-- Logs begin at Mon 2022-07-18 05:32:31 UTC, end at Mon 2022-07-18 07:33:33 UTC. -- Jul 18 06:04:50.202836 dv-cinder-850015-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 18 06:04:50.214931 dv-cinder-850015-4-iscsi devstack@keystone.service[80457]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 18 06:04:50.218441 dv-cinder-850015-4-iscsi devstack@keystone.service[80457]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jul 18 06:04:50 2022] *** Jul 18 06:04:50.218441 dv-cinder-850015-4-iscsi devstack@keystone.service[80457]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 18 06:04:50.218441 dv-cinder-850015-4-iscsi devstack@keystone.service[80457]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 18 06:04:50.218441 dv-cinder-850015-4-iscsi devstack@keystone.service[80457]: nodename: dv-cinder-850015-4-iscsi Jul 18 06:04:50.218441 dv-cinder-850015-4-iscsi devstack@keystone.service[80457]: machine: x86_64 Jul 18 06:04:50.218441 dv-cinder-850015-4-iscsi devstack@keystone.service[80457]: clock source: unix Jul 18 06:04:50.218441 dv-cinder-850015-4-iscsi devstack@keystone.service[80457]: pcre jit disabled Jul 18 06:04:50.218441 dv-cinder-850015-4-iscsi devstack@keystone.service[80457]: detected number of CPU cores: 4 Jul 18 06:04:50.218441 dv-cinder-850015-4-iscsi devstack@keystone.service[80457]: current working directory: / Jul 18 06:04:50.218441 dv-cinder-850015-4-iscsi devstack@keystone.service[80457]: detected binary path: /usr/bin/uwsgi-core Jul 18 06:04:50.219362 dv-cinder-850015-4-iscsi devstack@keystone.service[80457]: your processes number limit is 47680 Jul 18 06:04:50.219362 dv-cinder-850015-4-iscsi devstack@keystone.service[80457]: your memory page size is 4096 bytes Jul 18 06:04:50.219362 dv-cinder-850015-4-iscsi devstack@keystone.service[80457]: detected max file descriptor number: 2048 Jul 18 06:04:50.219362 dv-cinder-850015-4-iscsi devstack@keystone.service[80457]: lock engine: pthread robust mutexes Jul 18 06:04:50.219362 dv-cinder-850015-4-iscsi devstack@keystone.service[80457]: thunder lock: enabled Jul 18 06:04:50.219362 dv-cinder-850015-4-iscsi devstack@keystone.service[80457]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 18 06:04:50.219362 dv-cinder-850015-4-iscsi devstack@keystone.service[80457]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 18 06:04:50.260825 dv-cinder-850015-4-iscsi devstack@keystone.service[80457]: Python main interpreter initialized at 0x558a3b7469c0 Jul 18 06:04:50.260825 dv-cinder-850015-4-iscsi devstack@keystone.service[80457]: python threads support enabled Jul 18 06:04:50.260825 dv-cinder-850015-4-iscsi devstack@keystone.service[80457]: your server socket listen backlog is limited to 100 connections Jul 18 06:04:50.260825 dv-cinder-850015-4-iscsi devstack@keystone.service[80457]: your mercy for graceful operations on workers is 90 seconds Jul 18 06:04:50.261427 dv-cinder-850015-4-iscsi devstack@keystone.service[80457]: mapped 403077 bytes (393 KB) for 2 cores Jul 18 06:04:50.261427 dv-cinder-850015-4-iscsi devstack@keystone.service[80457]: *** Operational MODE: preforking *** Jul 18 06:04:50.261670 dv-cinder-850015-4-iscsi devstack@keystone.service[80457]: *** uWSGI is running in multiple interpreter mode *** Jul 18 06:04:50.261670 dv-cinder-850015-4-iscsi devstack@keystone.service[80457]: spawned uWSGI master process (pid: 80457) Jul 18 06:04:50.261778 dv-cinder-850015-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 18 06:04:50.266628 dv-cinder-850015-4-iscsi devstack@keystone.service[80457]: spawned uWSGI worker 1 (pid: 80470, cores: 1) Jul 18 06:04:50.266628 dv-cinder-850015-4-iscsi devstack@keystone.service[80457]: spawned uWSGI worker 2 (pid: 80471, cores: 1) Jul 18 06:04:50.266628 dv-cinder-850015-4-iscsi devstack@keystone.service[80457]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 18 06:04:56.129568 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 06:04:56.129568 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: from cryptography.utils import int_from_bytes Jul 18 06:04:56.130753 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 06:04:56.130753 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: from cryptography.utils import int_from_bytes Jul 18 06:04:56.188533 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 18 06:04:56.188782 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 18 06:04:56.189041 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] command line args: [] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 18 06:04:56.189380 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 18 06:04:56.189620 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ================================================================================ {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 18 06:04:56.189945 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] admin_token = **** {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.190257 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] config_dir = [] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.190509 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.190757 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] config_source = [] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.191000 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] debug = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.191265 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.191981 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.192179 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.192597 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.193029 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.193435 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] list_limit = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.193749 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] log_config_append = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.194063 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.194366 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] log_dir = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.194718 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] log_file = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.195088 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.195396 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.195699 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.195983 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.196260 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.196500 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.196791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.197728 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.198004 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.198290 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.198584 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.198873 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.199169 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.199480 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.199754 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.202306 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.40/identity {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.203025 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] publish_errors = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.203321 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.203610 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.203840 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.204144 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.204442 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.204750 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] standard_threads = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.205032 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.205316 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.205534 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.205757 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] use_journal = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.205975 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] use_json = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.206191 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] use_stderr = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.206386 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] use_syslog = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.206671 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.206975 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.207256 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.207568 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.207895 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.208188 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.208534 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.208830 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.209141 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.209436 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.209762 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.210057 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.210367 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.210653 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.210964 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.211253 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.211575 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.211860 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.212196 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.212535 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.212862 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.213139 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.213468 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.213776 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.214106 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.214395 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.214702 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.214977 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.215279 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.215558 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.216401 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.217411 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.217621 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 06:04:56.217621 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: from cryptography.utils import int_from_bytes Jul 18 06:04:56.218614 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.218614 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.219022 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 06:04:56.219022 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: from cryptography.utils import int_from_bytes Jul 18 06:04:56.219369 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.219369 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.219369 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.219369 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.220078 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.220078 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.220078 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.220634 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.220839 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.221018 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.221330 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.221630 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.221986 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.222257 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.222562 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.222792 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.223075 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.223354 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.223644 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.223917 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.224247 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.224573 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.224861 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.225145 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.225429 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.225710 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.226006 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.226289 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.226577 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.226857 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.227168 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.227452 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.227727 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.228011 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.228311 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.228714 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.228928 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.229189 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.229480 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.230469 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.230753 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.231072 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.231366 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.232345 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.233724 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.241843 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.242195 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.242523 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.242852 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.243177 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.243497 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.243918 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.244245 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.244602 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.244917 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.245231 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.245538 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.245844 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.246147 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.246445 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.246744 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.247045 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.247351 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.247650 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.247955 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.248400 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.248908 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.249341 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.249808 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.250282 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.250716 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.251143 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] auth.external = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.251585 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.252154 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.252632 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.253084 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] auth.password = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.253523 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] auth.token = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.253982 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.254432 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.254870 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.255111 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.255376 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.255638 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.255882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.256142 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] credential.caching = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.256399 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.256685 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.256943 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.257186 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.257438 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.257680 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.257959 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.258219 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.258461 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.258931 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.259212 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.259501 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.259909 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.260287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.260582 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.260828 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] federation.caching = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.261249 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.261703 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.262152 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.262598 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.263520 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.263520 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.264574 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.264574 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.265107 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.265107 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.265668 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.265976 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] identity.caching = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.266462 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.266927 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.267378 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.267834 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.268313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.268814 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.269274 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.269760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.270239 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.270697 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.271159 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.271614 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.272094 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.272584 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.273070 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.273578 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.274060 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.274589 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.275053 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.275281 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 18 06:04:56.275500 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 18 06:04:56.275728 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.275947 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] command line args: [] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 18 06:04:56.275947 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 18 06:04:56.276368 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.276602 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ================================================================================ {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 18 06:04:56.276602 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] admin_token = **** {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.277027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.277249 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] config_dir = [] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.277249 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.277647 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.277844 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] config_source = [] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.277844 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] debug = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.277844 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.278994 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.279146 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.279146 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.279146 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.279556 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.279693 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.279855 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.279993 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] list_limit = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.280127 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.280260 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] log_config_append = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.280260 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.280597 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.280741 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] log_dir = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.280886 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.281019 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] log_file = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.281149 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.281398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.281542 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.281803 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.282385 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.282526 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.282660 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.282795 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.282795 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.283063 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.283196 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.283330 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.283460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.283600 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.283600 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.283898 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.284040 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.284040 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.284277 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.284451 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.284451 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.284696 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.284829 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.284960 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.285091 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.285091 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.285347 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.285466 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.285581 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.285694 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.285809 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.286123 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.286265 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.286397 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.286528 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.286659 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.286792 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.user = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.286967 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.287238 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.287378 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.40/identity {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.287513 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.287656 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] publish_errors = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.287799 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.287990 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.288124 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.288256 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.288455 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.288855 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.288976 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.289100 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.289215 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.289450 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.289575 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.289705 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.289821 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.289945 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.290061 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.290317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] standard_threads = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.290463 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.290615 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.290744 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.290875 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.291006 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.291135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.291266 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.291462 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.291609 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.291752 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.291881 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] use_journal = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.292065 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.292213 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] use_json = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.292354 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.292354 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.292707 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] use_stderr = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.292854 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.292987 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] use_syslog = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.293118 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.293118 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.293356 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 06:04:56.293506 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.293506 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.293748 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.293880 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.294085 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.294698 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.295032 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.295252 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.295390 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.295520 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.295653 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.295856 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.296013 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.296148 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.296324 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.296498 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.296632 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] resource.caching = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.296836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.296995 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.297130 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.297260 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.297391 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.297551 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.297713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.297867 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.298037 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.298171 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.298303 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.298511 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.298659 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.298809 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.298975 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] role.caching = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.299112 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.299254 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] role.driver = sql {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.299435 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.299588 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.299835 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.299969 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.300104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.300247 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.300380 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.300627 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.300627 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.300888 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.301034 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.301171 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.301313 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.301456 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.301594 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.301722 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.301855 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.302055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.302192 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.302327 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.302521 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.302671 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.302918 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.303059 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.303192 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.303373 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.303512 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.303647 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.303840 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.303980 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.304121 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.304299 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.304543 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.304679 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.304810 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.304943 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.305074 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.305220 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.305354 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.305530 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.305665 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.305798 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.305987 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.306141 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.306278 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.306408 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.306537 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.306698 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.306858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.307010 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.307144 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.307313 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.307452 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] token.caching = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.307584 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.307713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.307857 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.308058 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.308194 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.308326 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.308570 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.308710 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.308862 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.308993 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.309136 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.309268 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.309432 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.309571 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.310336 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.310475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.310642 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.310833 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.311056 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.311232 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.311454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.311655 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.311807 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.311950 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.312085 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.312230 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.312493 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.312636 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.312768 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.connection = **** {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.312920 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.313067 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.313198 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.313325 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.313467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.313654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.313792 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.313935 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.314124 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.314261 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.314408 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.314573 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.314724 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.314858 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.315012 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.315152 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.315298 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.315428 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.315561 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.315724 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.315858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.315991 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.316189 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.316324 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.316477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.316674 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.316814 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.316947 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.317097 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 18 06:04:56.317224 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.317536 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.317892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.318243 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.318606 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.318953 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.319295 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.319692 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.320055 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.320391 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.320854 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.321229 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.321600 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.321935 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.322283 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.322630 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.323016 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.323359 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.323716 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.324077 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.324449 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.324800 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.325156 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.326095 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.326445 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.326789 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.327143 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.327490 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.327846 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.328240 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.328644 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.329033 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.329430 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.329782 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.330151 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.330519 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.330887 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.331287 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.331665 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.332021 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.332378 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.332856 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.333215 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.333604 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.333939 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.334279 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.334614 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.334970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.335304 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.335652 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.336008 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.336345 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.336720 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] auth.external = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.337053 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.337450 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.337785 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.338112 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] auth.password = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.338439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] auth.token = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.338821 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.339152 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.339493 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.339876 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.340221 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.340627 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.341012 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.341352 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] credential.caching = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.341698 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.342040 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.342384 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.342716 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.343060 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.343390 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.343753 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.344142 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.344515 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.344864 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.345258 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.345600 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.345961 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.346301 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.346654 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.346987 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] federation.caching = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.347326 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.347668 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.348012 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.348339 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.348713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.349049 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.349418 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.349752 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.350103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.350436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.350776 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.351107 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] identity.caching = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.351459 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.351804 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.352136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.352628 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.352972 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.353298 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.353641 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.353986 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.354340 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.354670 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.354996 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.355323 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.355539 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.355743 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.355887 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.356024 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.356172 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.357082 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.357438 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.357785 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.358150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.358493 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.358829 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.359166 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.359499 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.359953 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.360288 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.360653 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.360993 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.361352 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.361686 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.362031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.362398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.362734 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.363079 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.363426 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.363754 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.364086 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.364478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.364788 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.365126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.365383 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.365528 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.365853 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.366190 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.366538 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.366883 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.367218 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.367544 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.367916 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.368261 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.368640 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.368974 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.369317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.369645 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.user = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.369983 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.370338 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.370664 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.371009 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.371361 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.371708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.372041 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.372461 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.372792 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.373122 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.373465 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.373792 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.374134 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.374478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.374822 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.375166 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.375518 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.375847 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.376190 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.376563 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.376899 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.377231 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.377573 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.377914 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.378258 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.378591 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.378940 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.379430 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.379759 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.380052 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.380386 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.380746 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.381097 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.381444 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.381772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.382104 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.382440 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] resource.caching = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.382787 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.383135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.383474 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.383824 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.384167 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.384538 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.384883 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.385459 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.385670 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.386004 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] role.caching = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.386346 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] role.driver = sql {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.386671 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.387017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.387362 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.388290 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.388653 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.389001 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.389492 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.389790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.390135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.390474 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.390819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.391166 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.391510 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.391859 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.392386 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.392761 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.393110 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.393458 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.393805 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.394151 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.394482 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.394827 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.394979 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.395116 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.395278 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.395613 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.395948 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.396277 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.396634 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.396980 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.397333 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.397676 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.398022 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.398355 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.398683 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.399016 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] token.caching = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.399346 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.399698 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.400038 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.400390 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.400780 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.401115 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.401453 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.401793 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.402145 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.402479 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.402811 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.403142 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.404217 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.404630 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.405013 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.405357 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.405762 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.406117 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.connection = **** {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.406463 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.406815 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.407152 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.407490 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.407836 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.408171 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.408539 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.408872 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.409211 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.409541 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.409888 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.410218 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.410562 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.410896 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.411235 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.411560 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.411904 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.412238 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 06:04:56.412525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 18 06:04:56.440120 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.440496 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.440847 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.449840 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.471560 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.471801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.545116 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.600107 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.600354 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.604382 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.641172 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.641403 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.670544 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.670836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.670979 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.671187 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.671404 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.671619 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.686822 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.687248 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.687467 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.687698 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.688194 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.688460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.698879 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.699117 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.699324 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.699546 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.699764 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.699988 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.700309 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.704223 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.704475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.704715 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.704943 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.705156 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.705377 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.705598 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.712841 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.713629 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.726024 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.726263 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.726399 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.726534 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.726682 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.726797 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.726912 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.727042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.727171 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.727298 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.727408 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.727525 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.727649 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.727649 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 06:04:56.745144 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.745378 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 06:04:56.745594 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.745743 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.745898 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 06:04:56.746032 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 06:04:56.746191 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.746318 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.746442 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 06:04:56.746568 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.746693 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.746829 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.746958 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.747571 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.747699 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.747826 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.758809 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.759056 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.759198 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.759316 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.759428 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.759542 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.759780 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.759920 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.768739 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 06:04:56.769023 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80470) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 06:04:56.772043 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 06:04:56.772301 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80470) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 06:04:56.772473 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.772590 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.772720 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.772850 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.772964 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.773155 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.773290 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.773416 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.773541 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.773670 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.773810 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.773942 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.774060 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.774207 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.774327 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.774464 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.774592 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.774719 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.774842 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.775052 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.801734 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.802806 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.803384 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 06:04:56.803675 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.804477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80470) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 06:04:56.804764 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.809131 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 06:04:56.809411 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.810305 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.810640 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80470) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 06:04:56.811115 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.811471 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.817676 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 06:04:56.817969 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.818730 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80470) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 06:04:56.819019 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.822465 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 06:04:56.822755 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.823038 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.823325 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.823610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.823761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80470) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 06:04:56.823919 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.824058 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.824187 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.824323 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.824497 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.824632 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.824746 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.825023 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.825905 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.826367 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.843899 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.844196 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.845837 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.846128 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.848836 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 06:04:56.850747 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80470) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 06:04:56.852610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.852909 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.854427 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.854719 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.858640 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 06:04:56.860150 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80470) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 06:04:56.862310 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 06:04:56.863804 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80470) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 06:04:56.865957 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.866260 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.866544 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.866833 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.867130 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.867418 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.867647 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.867916 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.868119 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.868403 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.868738 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.869017 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.887058 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 06:04:56.887352 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.887626 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.888540 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.889711 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80470) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 06:04:56.890120 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.890393 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.890672 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.896611 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.898313 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.898590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.898744 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.900383 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.900688 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.901654 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 06:04:56.901940 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.902229 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.902516 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.902802 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.903796 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80470) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 06:04:56.904285 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.904681 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.904977 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.905254 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.916939 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 06:04:56.917233 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.917517 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.917799 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.918080 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.918367 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.918645 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.918936 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80470) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 06:04:56.919359 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.919818 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.920297 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.920635 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.920917 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.921194 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.937536 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 06:04:56.940888 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 06:04:56.941636 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80470) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 06:04:56.945001 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80470) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 06:04:56.945699 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.947340 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.948324 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.949018 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 06:04:56.949306 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.949587 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.949958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.952697 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80470) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 06:04:56.952982 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.953260 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.957492 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.959121 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 06:04:56.960694 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.962321 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80470) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 06:04:56.962601 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.962793 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.963081 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.963374 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.965701 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.966040 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.966469 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.966924 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.972876 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 06:04:56.973173 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.973459 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.976364 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80470) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 06:04:56.976692 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.976975 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.981596 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 06:04:56.981900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.982186 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.982463 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.982738 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.983028 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.983323 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.983597 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.983869 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.984471 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.984471 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.984721 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.984954 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.985223 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.988918 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80470) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 06:04:56.989429 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.989918 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.990398 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.990892 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.991396 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.991880 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.992372 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.992890 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.993378 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.993882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.994378 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.994826 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:56.995106 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 06:04:57.024268 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x558a3b7469c0 pid: 80471 (default app) Jul 18 06:04:57.027970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa51ae-18b3-4abb-b0f1-e0cf032370e4 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:04:57.028262 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa51ae-18b3-4abb-b0f1-e0cf032370e4 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:04:57.028774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa51ae-18b3-4abb-b0f1-e0cf032370e4 None None] PATH_INFO: `/v3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:04:57.033289 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1/1] 10.222.0.40 () {50 vars in 791 bytes} [Mon Jul 18 06:04:57 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:04:57.035182 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x558a3b7469c0 pid: 80470 (default app) Jul 18 06:05:06.609468 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c607b9-d96d-4498-b8d6-7d39babff8f3 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:06.610106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c607b9-d96d-4498-b8d6-7d39babff8f3 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:06.610106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c607b9-d96d-4498-b8d6-7d39babff8f3 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:06.611423 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 2/2] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:05:06.619977 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d95006-cad7-40c5-ae81-d2dc78ce3781 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:06.620609 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d95006-cad7-40c5-ae81-d2dc78ce3781 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:06.620609 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d95006-cad7-40c5-ae81-d2dc78ce3781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:06.693388 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: INFO dbcounter [None req-e8d95006-cad7-40c5-ae81-d2dc78ce3781 None None] Registered counter for database keystone Jul 18 06:05:06.717848 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG oslo_db.sqlalchemy.engines [None req-e8d95006-cad7-40c5-ae81-d2dc78ce3781 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80470) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 18 06:05:06.724861 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writer thread running {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 18 06:05:06.916285 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG passlib.handlers.bcrypt [None req-e8d95006-cad7-40c5-ae81-d2dc78ce3781 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80470) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 18 06:05:06.916756 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG passlib.handlers.bcrypt [None req-e8d95006-cad7-40c5-ae81-d2dc78ce3781 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80470) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 18 06:05:06.974409 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-e8d95006-cad7-40c5-ae81-d2dc78ce3781 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:06.976037 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e8d95006-cad7-40c5-ae81-d2dc78ce3781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:07.127538 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1/3] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:06 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 511 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 18 06:05:07.283061 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c607b9-d96d-4498-b8d6-7d39babff8f3 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:07.355278 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: INFO dbcounter [None req-80c607b9-d96d-4498-b8d6-7d39babff8f3 None None] Registered counter for database keystone Jul 18 06:05:07.382369 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG oslo_db.sqlalchemy.engines [None req-80c607b9-d96d-4498-b8d6-7d39babff8f3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80471) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 18 06:05:07.390269 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writer thread running {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 18 06:05:07.511024 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c607b9-d96d-4498-b8d6-7d39babff8f3 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:07.515851 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c79f856-f32f-446d-9850-9b6d4d3ae74c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:07.516550 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c79f856-f32f-446d-9850-9b6d4d3ae74c admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:07.516840 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c79f856-f32f-446d-9850-9b6d4d3ae74c admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:07.517116 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c79f856-f32f-446d-9850-9b6d4d3ae74c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:07.525912 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c79f856-f32f-446d-9850-9b6d4d3ae74c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:07.579427 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1c79f856-f32f-446d-9850-9b6d4d3ae74c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 18 06:05:07.579663 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c79f856-f32f-446d-9850-9b6d4d3ae74c admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:07.590199 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 3/4] 10.222.0.40 () {58 vars in 1324 bytes} [Mon Jul 18 06:05:07 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 06:05:07.596945 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d95006-cad7-40c5-ae81-d2dc78ce3781 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:07.614548 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d95006-cad7-40c5-ae81-d2dc78ce3781 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:07.619017 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dfb65d-6c15-48e8-96a0-57ed394a8330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:07.620147 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dfb65d-6c15-48e8-96a0-57ed394a8330 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:07.620742 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dfb65d-6c15-48e8-96a0-57ed394a8330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:07.620742 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dfb65d-6c15-48e8-96a0-57ed394a8330 admin admin] PATH_INFO: `/v3/projects/3e19cd5e83fe45f5a260277527d6bf8c` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:07.622472 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dfb65d-6c15-48e8-96a0-57ed394a8330 admin admin] RBAC: Authorizing `identity:get_project(project_id=3e19cd5e83fe45f5a260277527d6bf8c)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:07.673755 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c0dfb65d-6c15-48e8-96a0-57ed394a8330 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 18 06:05:07.674133 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dfb65d-6c15-48e8-96a0-57ed394a8330 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:07.678987 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 2/5] 10.222.0.40 () {56 vars in 1207 bytes} [Mon Jul 18 06:05:07 2022] GET /identity/v3/projects/3e19cd5e83fe45f5a260277527d6bf8c => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:05:09.741296 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fcff27-ec69-40e1-9846-415fa8238d88 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:09.741888 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fcff27-ec69-40e1-9846-415fa8238d88 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:09.741888 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fcff27-ec69-40e1-9846-415fa8238d88 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:09.746311 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 4/6] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:05:09.753567 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e16459-a4e7-4edd-bc6a-2f12322a9beb None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:09.754118 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e16459-a4e7-4edd-bc6a-2f12322a9beb None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:09.754118 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e16459-a4e7-4edd-bc6a-2f12322a9beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:09.775591 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-42e16459-a4e7-4edd-bc6a-2f12322a9beb None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:09.777349 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-42e16459-a4e7-4edd-bc6a-2f12322a9beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:09.786512 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 3/7] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:09 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 18 06:05:09.810721 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fcff27-ec69-40e1-9846-415fa8238d88 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:09.834539 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fcff27-ec69-40e1-9846-415fa8238d88 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:09.839189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e2faac-8bdb-4320-978b-2530b3c886fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:09.839866 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2faac-8bdb-4320-978b-2530b3c886fb admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:09.840162 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2faac-8bdb-4320-978b-2530b3c886fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:09.840512 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2faac-8bdb-4320-978b-2530b3c886fb admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:09.850387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e2faac-8bdb-4320-978b-2530b3c886fb admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:09.852223 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e2faac-8bdb-4320-978b-2530b3c886fb admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:09.863180 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 5/8] 10.222.0.40 () {58 vars in 1324 bytes} [Mon Jul 18 06:05:09 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 06:05:09.869739 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e16459-a4e7-4edd-bc6a-2f12322a9beb None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:09.887963 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e16459-a4e7-4edd-bc6a-2f12322a9beb None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:09.896060 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7d53c2-bdbd-483f-bad8-9ca4cd3405ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:09.897596 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d53c2-bdbd-483f-bad8-9ca4cd3405ca admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:09.898086 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d53c2-bdbd-483f-bad8-9ca4cd3405ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:09.898560 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d53c2-bdbd-483f-bad8-9ca4cd3405ca admin admin] PATH_INFO: `/v3/users/8783051891e642af84b75a46b4d6b15e` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:09.901018 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7d53c2-bdbd-483f-bad8-9ca4cd3405ca admin admin] RBAC: Authorizing `identity:get_user(user_id=8783051891e642af84b75a46b4d6b15e)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:09.905608 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7d53c2-bdbd-483f-bad8-9ca4cd3405ca admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:09.908677 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 4/9] 10.222.0.40 () {56 vars in 1198 bytes} [Mon Jul 18 06:05:09 2022] GET /identity/v3/users/8783051891e642af84b75a46b4d6b15e => generated 247 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:12.051061 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec9bb5-382b-4fd6-8d99-122d59311250 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:12.051574 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec9bb5-382b-4fd6-8d99-122d59311250 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:12.051994 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec9bb5-382b-4fd6-8d99-122d59311250 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:12.054655 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 6/10] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:05:12.063178 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a46c6-829b-4fa6-87a4-19625c555128 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:12.064364 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a46c6-829b-4fa6-87a4-19625c555128 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:12.064364 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a46c6-829b-4fa6-87a4-19625c555128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:12.080218 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68416d3e-8731-47a0-8c0f-33f775e36130 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:12.080547 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68416d3e-8731-47a0-8c0f-33f775e36130 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:12.080833 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68416d3e-8731-47a0-8c0f-33f775e36130 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:12.082212 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 7/11] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:05:12.082785 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-590a46c6-829b-4fa6-87a4-19625c555128 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:12.084138 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-590a46c6-829b-4fa6-87a4-19625c555128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:12.089025 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3d5a8-eaec-4c7a-ac9b-945c1e24538e None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:12.089434 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3d5a8-eaec-4c7a-ac9b-945c1e24538e None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:12.090015 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3d5a8-eaec-4c7a-ac9b-945c1e24538e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:12.132716 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG passlib.handlers.bcrypt [None req-e4b3d5a8-eaec-4c7a-ac9b-945c1e24538e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80471) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 18 06:05:12.133226 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG passlib.handlers.bcrypt [None req-e4b3d5a8-eaec-4c7a-ac9b-945c1e24538e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80471) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 18 06:05:12.173403 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 5/12] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:05:12.180745 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-e4b3d5a8-eaec-4c7a-ac9b-945c1e24538e None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:12.182365 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e4b3d5a8-eaec-4c7a-ac9b-945c1e24538e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:12.196880 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a46c6-829b-4fa6-87a4-19625c555128 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:12.223537 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:12.298303 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 8/13] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:05:12.321441 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b3d5a8-eaec-4c7a-ac9b-945c1e24538e None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:12.326574 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a46c6-829b-4fa6-87a4-19625c555128 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:12.366818 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7fb53-1954-4055-a5a8-7e2cc5813456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:12.367639 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7fb53-1954-4055-a5a8-7e2cc5813456 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:12.367932 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7fb53-1954-4055-a5a8-7e2cc5813456 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:12.368224 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7fb53-1954-4055-a5a8-7e2cc5813456 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:12.381700 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd7fb53-1954-4055-a5a8-7e2cc5813456 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:05:12.393419 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-3dd7fb53-1954-4055-a5a8-7e2cc5813456 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:05:12.393419 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:05:12.393419 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:05:12.393419 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:05:12.393419 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:05:12.393419 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:05:12.393419 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:05:12.393419 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:05:12.393419 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:05:12.393419 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:05:12.393419 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:05:12.393419 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:05:12.393419 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:05:12.393419 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:05:12.393419 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:05:12.393419 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:05:12.394427 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:05:12.394427 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:05:12.394427 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:05:12.394427 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:05:12.394427 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:05:12.394427 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:05:12.394427 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:05:12.394427 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:05:12.394427 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:05:12.394427 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:05:12.394427 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:05:12.394427 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:05:12.394427 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:05:12.394427 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:05:12.394427 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:05:12.394427 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:05:12.395338 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:05:12.395338 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:05:12.395338 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:05:12.395338 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:05:12.395338 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:05:12.395338 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:05:12.395338 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:05:12.395338 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:05:12.395338 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:05:12.395338 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:05:12.395338 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:05:12.395338 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:05:12.396750 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 6/14] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 18 06:05:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:05:12.401177 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7fb53-1954-4055-a5a8-7e2cc5813456 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:12.433870 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b3d5a8-eaec-4c7a-ac9b-945c1e24538e None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:12.437314 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:12.492187 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4809d825-e936-4140-9100-be268826c840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:12.493527 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809d825-e936-4140-9100-be268826c840 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:12.494046 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809d825-e936-4140-9100-be268826c840 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:12.494555 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809d825-e936-4140-9100-be268826c840 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:12.503404 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:12.525162 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7fb53-1954-4055-a5a8-7e2cc5813456 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:12.582709 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3aeaf1-eaa6-4ca1-bb92-4a5ae2e44314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:12.583493 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3aeaf1-eaa6-4ca1-bb92-4a5ae2e44314 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:12.583818 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3aeaf1-eaa6-4ca1-bb92-4a5ae2e44314 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:12.584109 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3aeaf1-eaa6-4ca1-bb92-4a5ae2e44314 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:12.585051 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3aeaf1-eaa6-4ca1-bb92-4a5ae2e44314 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:12.585687 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3aeaf1-eaa6-4ca1-bb92-4a5ae2e44314 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:12.587090 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3aeaf1-eaa6-4ca1-bb92-4a5ae2e44314 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:12.602109 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 7/15] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:05:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:05:12.607916 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4809d825-e936-4140-9100-be268826c840 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:12.608765 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3aeaf1-eaa6-4ca1-bb92-4a5ae2e44314 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:12.609532 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4809d825-e936-4140-9100-be268826c840 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:12.736545 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 9/16] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:05:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 06:05:12.736767 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3aeaf1-eaa6-4ca1-bb92-4a5ae2e44314 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:12.737627 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4809d825-e936-4140-9100-be268826c840 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:12.740630 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:12.769074 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f3ebd-e3a6-40f1-b2c6-0625a8ebb396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:12.770053 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f3ebd-e3a6-40f1-b2c6-0625a8ebb396 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:12.770341 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f3ebd-e3a6-40f1-b2c6-0625a8ebb396 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:12.770623 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f3ebd-e3a6-40f1-b2c6-0625a8ebb396 None admin] PATH_INFO: `/v3/users/8783051891e642af84b75a46b4d6b15e` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:12.771866 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70f3ebd-e3a6-40f1-b2c6-0625a8ebb396 None admin] RBAC: Authorizing `identity:get_user(user_id=8783051891e642af84b75a46b4d6b15e)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:12.773409 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70f3ebd-e3a6-40f1-b2c6-0625a8ebb396 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:12.775217 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 8/17] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:05:12 2022] GET /identity/v3/users/8783051891e642af84b75a46b4d6b15e => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:05:12.781707 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f3ebd-e3a6-40f1-b2c6-0625a8ebb396 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:12.799346 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:12.850969 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4809d825-e936-4140-9100-be268826c840 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:12.890031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b982a3e4-062a-4dcf-a18c-08c3d6b44640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:12.890854 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982a3e4-062a-4dcf-a18c-08c3d6b44640 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:12.891145 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982a3e4-062a-4dcf-a18c-08c3d6b44640 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:12.891429 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982a3e4-062a-4dcf-a18c-08c3d6b44640 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:12.904034 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f3ebd-e3a6-40f1-b2c6-0625a8ebb396 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:12.914455 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b982a3e4-062a-4dcf-a18c-08c3d6b44640 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:12.915979 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b982a3e4-062a-4dcf-a18c-08c3d6b44640 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:12.940900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-b982a3e4-062a-4dcf-a18c-08c3d6b44640 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:05:12.940900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:05:12.940900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:05:12.940900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:05:12.940900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:05:12.940900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:05:12.940900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:05:12.940900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:05:12.940900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:05:12.940900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:05:12.940900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:05:12.940900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:05:12.940900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:05:12.940900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:05:12.940900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:05:12.942525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:05:12.942525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:05:12.942525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:05:12.942525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:05:12.942525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:05:12.942525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:05:12.942525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:05:12.942525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:05:12.942525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:05:12.942525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:05:12.942525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:05:12.942525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:05:12.942525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:05:12.942525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:05:12.942525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:05:12.942525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:05:12.942525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:05:12.944028 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:05:12.944028 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:05:12.944028 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:05:12.944028 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:05:12.944028 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 10/18] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:05:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:05:12.945397 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6fdbf6-7f16-491a-87ff-3217adeb5f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:12.946229 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fdbf6-7f16-491a-87ff-3217adeb5f8d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:12.946520 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fdbf6-7f16-491a-87ff-3217adeb5f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:12.946803 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fdbf6-7f16-491a-87ff-3217adeb5f8d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:12.948095 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6fdbf6-7f16-491a-87ff-3217adeb5f8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:12.948298 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b982a3e4-062a-4dcf-a18c-08c3d6b44640 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:12.949645 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6fdbf6-7f16-491a-87ff-3217adeb5f8d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:12.951911 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 9/19] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:05:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:05:12.961482 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6fdbf6-7f16-491a-87ff-3217adeb5f8d None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:13.017003 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:13.058339 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:13.070437 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b982a3e4-062a-4dcf-a18c-08c3d6b44640 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:13.082975 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6fdbf6-7f16-491a-87ff-3217adeb5f8d None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:13.105703 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7681cc-a9dd-46a4-9996-7314810456a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:13.106778 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7681cc-a9dd-46a4-9996-7314810456a0 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:13.107064 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7681cc-a9dd-46a4-9996-7314810456a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:13.107349 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7681cc-a9dd-46a4-9996-7314810456a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:13.108084 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7681cc-a9dd-46a4-9996-7314810456a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:13.108400 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7681cc-a9dd-46a4-9996-7314810456a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:13.109814 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7681cc-a9dd-46a4-9996-7314810456a0 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:13.128728 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 11/20] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:05:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:05:13.135640 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92788e32-0387-4753-8b89-ddf2e7d5b27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:13.136338 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92788e32-0387-4753-8b89-ddf2e7d5b27e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:13.136660 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92788e32-0387-4753-8b89-ddf2e7d5b27e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:13.136945 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92788e32-0387-4753-8b89-ddf2e7d5b27e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:13.137846 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92788e32-0387-4753-8b89-ddf2e7d5b27e None admin] RBAC: Adding query filter params (role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc, scope.domain.id=default, user.id=8783051891e642af84b75a46b4d6b15e) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:13.138171 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92788e32-0387-4753-8b89-ddf2e7d5b27e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:13.139841 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92788e32-0387-4753-8b89-ddf2e7d5b27e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:13.148181 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 10/21] 10.222.0.40 () {56 vars in 1322 bytes} [Mon Jul 18 06:05:12 2022] GET /identity/v3/role_assignments?user.id=8783051891e642af84b75a46b4d6b15e&scope.domain.id=default&role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 233 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:05:15.568320 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06341d43-ebc0-4b06-a772-51fa71f36371 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:15.569555 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06341d43-ebc0-4b06-a772-51fa71f36371 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:15.570111 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06341d43-ebc0-4b06-a772-51fa71f36371 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:15.572517 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 12/22] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:05:15.577934 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd5b51-c8cd-4318-95b9-b4e936698065 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:15.578877 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd5b51-c8cd-4318-95b9-b4e936698065 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:15.579383 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd5b51-c8cd-4318-95b9-b4e936698065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:15.606616 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-07dd5b51-c8cd-4318-95b9-b4e936698065 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:15.608241 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-07dd5b51-c8cd-4318-95b9-b4e936698065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:15.713888 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 11/23] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:05:15.744515 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06341d43-ebc0-4b06-a772-51fa71f36371 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:15.894403 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06341d43-ebc0-4b06-a772-51fa71f36371 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:15.917535 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:15.937090 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7604dabe-44fc-41c2-8c02-5472afd048bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:15.937910 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604dabe-44fc-41c2-8c02-5472afd048bb None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:15.938199 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604dabe-44fc-41c2-8c02-5472afd048bb None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:15.938484 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604dabe-44fc-41c2-8c02-5472afd048bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:15.954729 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7604dabe-44fc-41c2-8c02-5472afd048bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:15.956584 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7604dabe-44fc-41c2-8c02-5472afd048bb None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:15.978704 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-7604dabe-44fc-41c2-8c02-5472afd048bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:05:15.978704 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:05:15.978704 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:05:15.978704 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:05:15.978704 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:05:15.978704 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:05:15.978704 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:05:15.978704 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:05:15.978704 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:05:15.978704 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:05:15.978704 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:05:15.978704 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:05:15.978704 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:05:15.978704 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:05:15.978704 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:05:15.979583 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:05:15.979583 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:05:15.979583 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:05:15.979583 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:05:15.979583 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:05:15.979583 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:05:15.979583 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:05:15.979583 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:05:15.979583 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:05:15.979583 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:05:15.979583 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:05:15.979583 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:05:15.979583 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:05:15.979583 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:05:15.979583 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:05:15.979583 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:05:15.979583 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:05:15.980397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:05:15.980397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:05:15.980397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:05:15.980397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:05:15.980397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 13/24] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:05:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:05:15.989520 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd5b51-c8cd-4318-95b9-b4e936698065 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:16.077148 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:16.110491 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd5b51-c8cd-4318-95b9-b4e936698065 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:16.158342 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ecfeb-16bb-486c-9bfa-f5457240056c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:16.158949 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ecfeb-16bb-486c-9bfa-f5457240056c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:16.159393 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ecfeb-16bb-486c-9bfa-f5457240056c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:16.159812 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ecfeb-16bb-486c-9bfa-f5457240056c None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:16.160839 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ecfeb-16bb-486c-9bfa-f5457240056c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:16.161452 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ecfeb-16bb-486c-9bfa-f5457240056c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:16.163101 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ecfeb-16bb-486c-9bfa-f5457240056c None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:16.199407 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 12/25] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:05:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:16.207750 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7604dabe-44fc-41c2-8c02-5472afd048bb None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:16.382301 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7604dabe-44fc-41c2-8c02-5472afd048bb None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:16.440321 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73164361-f091-4e14-91a9-4b997a735ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:16.441400 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73164361-f091-4e14-91a9-4b997a735ae6 None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:16.441928 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73164361-f091-4e14-91a9-4b997a735ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:16.442353 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73164361-f091-4e14-91a9-4b997a735ae6 None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:16.443493 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73164361-f091-4e14-91a9-4b997a735ae6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:16.445294 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73164361-f091-4e14-91a9-4b997a735ae6 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:16.529537 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91103382-b52a-4894-807c-25ad0b8affb6 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:16.530093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91103382-b52a-4894-807c-25ad0b8affb6 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:16.530514 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91103382-b52a-4894-807c-25ad0b8affb6 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:16.532059 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 13/26] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:05:16.537920 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206d48b-d216-43c1-9257-4f1d682d2cdd None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:16.538801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206d48b-d216-43c1-9257-4f1d682d2cdd None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:16.539248 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206d48b-d216-43c1-9257-4f1d682d2cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:16.559716 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-0206d48b-d216-43c1-9257-4f1d682d2cdd None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:16.561374 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-0206d48b-d216-43c1-9257-4f1d682d2cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:16.729924 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 14/27] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 06:05:16.731752 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70260a-0f79-4011-9e0a-47dbd557f035 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:16.732050 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70260a-0f79-4011-9e0a-47dbd557f035 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:16.732330 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70260a-0f79-4011-9e0a-47dbd557f035 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:16.734008 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 15/28] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:05:16.746157 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3762e89d-b855-4eda-968f-401389ea5d86 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:16.746500 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3762e89d-b855-4eda-968f-401389ea5d86 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:16.747278 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3762e89d-b855-4eda-968f-401389ea5d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:16.748520 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 14/29] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Jul 18 06:05:16 2022] POST /identity/v3/projects => generated 301 bytes in 541 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:05:16.770015 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73164361-f091-4e14-91a9-4b997a735ae6 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:16.788152 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-3762e89d-b855-4eda-968f-401389ea5d86 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:16.790933 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-3762e89d-b855-4eda-968f-401389ea5d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:16.812816 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:16.960204 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:16.971996 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 16/30] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 06:05:16.975121 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5169b4-8856-451d-a115-49c7943a8851 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:16.975414 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5169b4-8856-451d-a115-49c7943a8851 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:16.975696 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5169b4-8856-451d-a115-49c7943a8851 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:16.977156 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 17/31] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:05:16.987285 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0c135-6754-4f91-828f-10331ebf56fd None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:16.987557 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0c135-6754-4f91-828f-10331ebf56fd None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:16.987765 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0c135-6754-4f91-828f-10331ebf56fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:17.013521 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73164361-f091-4e14-91a9-4b997a735ae6 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:17.033023 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-4fb0c135-6754-4f91-828f-10331ebf56fd None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:17.035885 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4fb0c135-6754-4f91-828f-10331ebf56fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:17.080878 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af0273c-eb72-4d74-8ce2-9e38f1d51351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:17.081730 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0273c-eb72-4d74-8ce2-9e38f1d51351 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:17.082082 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0273c-eb72-4d74-8ce2-9e38f1d51351 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:17.082377 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0273c-eb72-4d74-8ce2-9e38f1d51351 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:17.097731 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af0273c-eb72-4d74-8ce2-9e38f1d51351 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:05:17.108287 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-5af0273c-eb72-4d74-8ce2-9e38f1d51351 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:05:17.108287 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:05:17.108287 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:05:17.108287 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:05:17.108287 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:05:17.108287 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:05:17.108287 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:05:17.108287 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:05:17.108287 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:05:17.108287 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:05:17.108287 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:05:17.108287 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:05:17.108287 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:05:17.108287 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:05:17.108287 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:05:17.108287 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:05:17.109446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:05:17.109446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:05:17.109446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:05:17.109446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:05:17.109446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:05:17.109446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:05:17.109446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:05:17.109446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:05:17.109446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:05:17.109446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:05:17.109446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:05:17.109446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:05:17.109446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:05:17.109446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:05:17.109446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:05:17.109446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:05:17.110395 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:05:17.110395 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:05:17.110395 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:05:17.110395 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:05:17.110395 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:05:17.110395 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:05:17.110395 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:05:17.110395 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:05:17.110395 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:05:17.110395 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:05:17.110395 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:05:17.110395 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:05:17.113104 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 15/32] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 18 06:05:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:05:17.114622 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af0273c-eb72-4d74-8ce2-9e38f1d51351 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:17.172695 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 18/33] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:05:17.174055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb0c135-6754-4f91-828f-10331ebf56fd None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:17.289048 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:17.302090 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb0c135-6754-4f91-828f-10331ebf56fd None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:17.333787 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:17.338656 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af0273c-eb72-4d74-8ce2-9e38f1d51351 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:17.348297 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d79f04-69c6-4640-961f-7489e7f3296e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:17.349361 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d79f04-69c6-4640-961f-7489e7f3296e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:17.349845 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d79f04-69c6-4640-961f-7489e7f3296e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:17.350344 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d79f04-69c6-4640-961f-7489e7f3296e None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:17.351288 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d79f04-69c6-4640-961f-7489e7f3296e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:17.351762 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d79f04-69c6-4640-961f-7489e7f3296e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:17.353834 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d79f04-69c6-4640-961f-7489e7f3296e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:17.365075 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 19/34] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:05:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:17.366675 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d79f04-69c6-4640-961f-7489e7f3296e None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:17.419617 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a773365-362b-4789-a9aa-e5f32cfb2206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:17.421302 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a773365-362b-4789-a9aa-e5f32cfb2206 None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:17.422427 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a773365-362b-4789-a9aa-e5f32cfb2206 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:17.423505 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a773365-362b-4789-a9aa-e5f32cfb2206 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:17.425400 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a773365-362b-4789-a9aa-e5f32cfb2206 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:17.430008 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a773365-362b-4789-a9aa-e5f32cfb2206 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:17.496568 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d79f04-69c6-4640-961f-7489e7f3296e None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:17.540479 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 16/35] 10.222.0.40 () {60 vars in 1131 bytes} [Mon Jul 18 06:05:17 2022] POST /identity/v3/roles => generated 219 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:05:17.542691 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cfb5b2-91be-4583-b01f-a3e598c22cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:17.543156 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a773365-362b-4789-a9aa-e5f32cfb2206 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:17.544320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cfb5b2-91be-4583-b01f-a3e598c22cb3 None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:17.544685 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cfb5b2-91be-4583-b01f-a3e598c22cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:17.545118 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cfb5b2-91be-4583-b01f-a3e598c22cb3 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:17.546043 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cfb5b2-91be-4583-b01f-a3e598c22cb3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:17.547674 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cfb5b2-91be-4583-b01f-a3e598c22cb3 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:17.707901 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a773365-362b-4789-a9aa-e5f32cfb2206 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:17.749002 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2db47d2-2452-4ead-9ed4-fbc61c7c2071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:17.749889 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db47d2-2452-4ead-9ed4-fbc61c7c2071 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:17.750177 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db47d2-2452-4ead-9ed4-fbc61c7c2071 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:17.750458 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db47d2-2452-4ead-9ed4-fbc61c7c2071 None admin] PATH_INFO: `/v3/users/8783051891e642af84b75a46b4d6b15e` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:17.751993 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2db47d2-2452-4ead-9ed4-fbc61c7c2071 None admin] RBAC: Authorizing `identity:get_user(user_id=8783051891e642af84b75a46b4d6b15e)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:17.753482 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2db47d2-2452-4ead-9ed4-fbc61c7c2071 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:17.755168 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 17/36] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:05:17 2022] GET /identity/v3/users/8783051891e642af84b75a46b4d6b15e => generated 247 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:17.756563 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98924e28-0b68-4dd6-b421-92d8c081d563 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:17.756930 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98924e28-0b68-4dd6-b421-92d8c081d563 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:17.757223 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98924e28-0b68-4dd6-b421-92d8c081d563 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:17.758572 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 18/37] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:05:17.759569 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c70991-c453-4acf-8043-c38cb30d8be9 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:17.759860 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c70991-c453-4acf-8043-c38cb30d8be9 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:17.760144 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c70991-c453-4acf-8043-c38cb30d8be9 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:17.761763 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 19/38] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:05:17.764761 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c70991-c453-4acf-8043-c38cb30d8be9 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:17.777097 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:17.856545 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 20/39] 10.222.0.40 () {60 vars in 1131 bytes} [Mon Jul 18 06:05:17 2022] POST /identity/v3/roles => generated 225 bytes in 480 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:05:17.856545 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c728dd-34e4-4b3e-be7c-53f89c6a441a None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:17.856545 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c728dd-34e4-4b3e-be7c-53f89c6a441a None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:17.856545 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c728dd-34e4-4b3e-be7c-53f89c6a441a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:17.877166 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-04c728dd-34e4-4b3e-be7c-53f89c6a441a None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:17.878802 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-04c728dd-34e4-4b3e-be7c-53f89c6a441a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:17.884772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c70991-c453-4acf-8043-c38cb30d8be9 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:17.939719 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:17.946308 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cdc5c3-a372-4d24-a5ea-fb953f96efb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:17.947705 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cdc5c3-a372-4d24-a5ea-fb953f96efb9 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:17.954487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cdc5c3-a372-4d24-a5ea-fb953f96efb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:17.954487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cdc5c3-a372-4d24-a5ea-fb953f96efb9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:17.954487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cdc5c3-a372-4d24-a5ea-fb953f96efb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:17.954487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cdc5c3-a372-4d24-a5ea-fb953f96efb9 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:17.957770 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 20/40] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:05:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:05:17.968685 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca567df-4bc6-4561-b272-545e3808f4e4 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:17.969219 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca567df-4bc6-4561-b272-545e3808f4e4 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:17.969596 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca567df-4bc6-4561-b272-545e3808f4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:17.972055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 21/41] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:05:17.972657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c728dd-34e4-4b3e-be7c-53f89c6a441a None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:18.002044 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-6ca567df-4bc6-4561-b272-545e3808f4e4 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:18.003667 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-6ca567df-4bc6-4561-b272-545e3808f4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:18.112548 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 21/42] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:05:18.113735 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca567df-4bc6-4561-b272-545e3808f4e4 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:18.128876 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:18.202003 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c728dd-34e4-4b3e-be7c-53f89c6a441a None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:18.237945 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2a8b00-4c29-4d47-b2ba-6bedb2218370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:18.238836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a8b00-4c29-4d47-b2ba-6bedb2218370 None admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:18.239127 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a8b00-4c29-4d47-b2ba-6bedb2218370 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:18.239427 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a8b00-4c29-4d47-b2ba-6bedb2218370 None admin] PATH_INFO: `/v3/domains/default/users/8783051891e642af84b75a46b4d6b15e/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:18.241531 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2a8b00-4c29-4d47-b2ba-6bedb2218370 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=8783051891e642af84b75a46b4d6b15e, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:18.243155 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2a8b00-4c29-4d47-b2ba-6bedb2218370 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:18.254304 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca567df-4bc6-4561-b272-545e3808f4e4 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:18.297162 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe2f647-3919-4a18-b021-9011a971e3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:18.298038 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe2f647-3919-4a18-b021-9011a971e3a0 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:18.298328 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe2f647-3919-4a18-b021-9011a971e3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:18.298604 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe2f647-3919-4a18-b021-9011a971e3a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:18.299830 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe2f647-3919-4a18-b021-9011a971e3a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:18.301337 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe2f647-3919-4a18-b021-9011a971e3a0 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:18.303022 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 22/43] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:05:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:18.303810 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe2f647-3919-4a18-b021-9011a971e3a0 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:18.324393 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 22/44] 10.222.0.40 () {58 vars in 1361 bytes} [Mon Jul 18 06:05:17 2022] PUT /identity/v3/domains/default/users/8783051891e642af84b75a46b4d6b15e/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:05:18.326455 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2a8b00-4c29-4d47-b2ba-6bedb2218370 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:18.375575 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:18.442446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:18.472437 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2a8b00-4c29-4d47-b2ba-6bedb2218370 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:18.477012 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe2f647-3919-4a18-b021-9011a971e3a0 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:18.534755 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03825267-425e-4964-b876-6a3adc40cfda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:18.535958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03825267-425e-4964-b876-6a3adc40cfda None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:18.536585 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03825267-425e-4964-b876-6a3adc40cfda None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:18.537035 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03825267-425e-4964-b876-6a3adc40cfda None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:18.538675 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03825267-425e-4964-b876-6a3adc40cfda None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:18.539320 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a67ddb-30de-4164-8066-8d81399032f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:18.540364 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a67ddb-30de-4164-8066-8d81399032f8 None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:18.540874 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a67ddb-30de-4164-8066-8d81399032f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:18.541342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a67ddb-30de-4164-8066-8d81399032f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:18.541447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03825267-425e-4964-b876-6a3adc40cfda None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:18.542714 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a67ddb-30de-4164-8066-8d81399032f8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:18.545216 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a67ddb-30de-4164-8066-8d81399032f8 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:18.621315 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 23/45] 10.222.0.40 () {60 vars in 1131 bytes} [Mon Jul 18 06:05:18 2022] POST /identity/v3/roles => generated 223 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:05:18.653136 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 23/46] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Jul 18 06:05:18 2022] POST /identity/v3/projects => generated 312 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:05:20.460680 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081e2f5-477a-4e81-b8fc-ce07dc7f123b None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:20.461475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081e2f5-477a-4e81-b8fc-ce07dc7f123b None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:20.461618 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081e2f5-477a-4e81-b8fc-ce07dc7f123b None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:20.463215 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 24/47] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:05:20.469432 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91575c29-f6bc-49c2-8393-7e7d2c3d70a9 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:20.469900 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91575c29-f6bc-49c2-8393-7e7d2c3d70a9 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:20.470016 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91575c29-f6bc-49c2-8393-7e7d2c3d70a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:20.487486 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-91575c29-f6bc-49c2-8393-7e7d2c3d70a9 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:20.488736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-91575c29-f6bc-49c2-8393-7e7d2c3d70a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:20.563425 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 24/48] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 06:05:20.586886 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f081e2f5-477a-4e81-b8fc-ce07dc7f123b None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:20.721958 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f081e2f5-477a-4e81-b8fc-ce07dc7f123b None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:20.759370 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a72ceda-29df-411e-b804-245ca3944764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:20.760207 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a72ceda-29df-411e-b804-245ca3944764 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:20.760521 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a72ceda-29df-411e-b804-245ca3944764 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:20.760809 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a72ceda-29df-411e-b804-245ca3944764 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:20.763610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:20.773973 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a72ceda-29df-411e-b804-245ca3944764 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:05:20.782538 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-0a72ceda-29df-411e-b804-245ca3944764 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:05:20.782538 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:05:20.782538 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:05:20.782538 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:05:20.782538 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:05:20.782538 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:05:20.782538 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:05:20.782538 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:05:20.782538 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:05:20.782538 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:05:20.782538 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:05:20.782538 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:05:20.782538 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:05:20.782538 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:05:20.782538 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:05:20.782538 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:05:20.783497 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:05:20.783497 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:05:20.783497 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:05:20.783497 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:05:20.783497 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:05:20.783497 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:05:20.783497 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:05:20.783497 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:05:20.783497 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:05:20.783497 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:05:20.783497 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:05:20.783497 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:05:20.783497 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:05:20.783497 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:05:20.783497 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:05:20.783497 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:05:20.784272 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:05:20.784272 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:05:20.784272 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:05:20.784272 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:05:20.784272 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:05:20.784272 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:05:20.784272 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:05:20.784272 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:05:20.784272 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:05:20.784272 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:05:20.784272 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:05:20.784272 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:05:20.784272 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 25/49] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 18 06:05:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:05:20.788026 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9a7e8-c4aa-4920-82cd-8dfada019bb0 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:20.788659 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9a7e8-c4aa-4920-82cd-8dfada019bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:20.789040 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9a7e8-c4aa-4920-82cd-8dfada019bb0 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:20.792585 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a72ceda-29df-411e-b804-245ca3944764 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:20.792786 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 25/50] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:05:20.797552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd93a43-ebe5-4d47-b9f6-73a240b81104 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:20.797954 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd93a43-ebe5-4d47-b9f6-73a240b81104 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:20.798339 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd93a43-ebe5-4d47-b9f6-73a240b81104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:20.817006 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-5dd93a43-ebe5-4d47-b9f6-73a240b81104 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:20.818230 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-5dd93a43-ebe5-4d47-b9f6-73a240b81104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:20.833573 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:20.894536 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 26/51] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 06:05:20.902841 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a72ceda-29df-411e-b804-245ca3944764 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:20.917387 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd93a43-ebe5-4d47-b9f6-73a240b81104 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:20.944154 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b60f1-eb06-4f84-a513-aac6ea7ccff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:20.944897 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b60f1-eb06-4f84-a513-aac6ea7ccff3 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:20.945303 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b60f1-eb06-4f84-a513-aac6ea7ccff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:20.945590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b60f1-eb06-4f84-a513-aac6ea7ccff3 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:20.946553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b60f1-eb06-4f84-a513-aac6ea7ccff3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:20.946873 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b60f1-eb06-4f84-a513-aac6ea7ccff3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:20.948346 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b60f1-eb06-4f84-a513-aac6ea7ccff3 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:20.964062 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 26/52] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:05:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:05:20.972919 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b60f1-eb06-4f84-a513-aac6ea7ccff3 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:21.040125 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd93a43-ebe5-4d47-b9f6-73a240b81104 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:21.041742 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:21.081012 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4bfe3f-b551-414f-b4b3-e448a30452ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:21.081829 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4bfe3f-b551-414f-b4b3-e448a30452ad None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:21.082124 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4bfe3f-b551-414f-b4b3-e448a30452ad None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:21.082411 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4bfe3f-b551-414f-b4b3-e448a30452ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:21.083776 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4bfe3f-b551-414f-b4b3-e448a30452ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:21.085319 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4bfe3f-b551-414f-b4b3-e448a30452ad None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:21.087013 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 27/53] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:05:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:05:21.092926 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b60f1-eb06-4f84-a513-aac6ea7ccff3 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:21.093106 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4bfe3f-b551-414f-b4b3-e448a30452ad None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:21.113839 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:21.213034 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4bfe3f-b551-414f-b4b3-e448a30452ad None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:21.239291 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc53bd-8a11-4dfa-bd89-1196dd588bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:21.240085 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc53bd-8a11-4dfa-bd89-1196dd588bbe None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:21.240377 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc53bd-8a11-4dfa-bd89-1196dd588bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:21.240694 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc53bd-8a11-4dfa-bd89-1196dd588bbe None admin] PATH_INFO: `/v3/users/8783051891e642af84b75a46b4d6b15e` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:21.241974 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc53bd-8a11-4dfa-bd89-1196dd588bbe None admin] RBAC: Authorizing `identity:get_user(user_id=8783051891e642af84b75a46b4d6b15e)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:21.243514 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc53bd-8a11-4dfa-bd89-1196dd588bbe None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:21.245540 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 27/54] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:05:20 2022] GET /identity/v3/users/8783051891e642af84b75a46b4d6b15e => generated 247 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:05:21.252129 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc53bd-8a11-4dfa-bd89-1196dd588bbe None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:21.255411 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcc3fad-d444-4b70-a823-e9ad0f060732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:21.256259 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc3fad-d444-4b70-a823-e9ad0f060732 None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:21.256778 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc3fad-d444-4b70-a823-e9ad0f060732 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:21.257266 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc3fad-d444-4b70-a823-e9ad0f060732 None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:21.258121 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcc3fad-d444-4b70-a823-e9ad0f060732 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:21.259654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcc3fad-d444-4b70-a823-e9ad0f060732 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:21.367042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 28/55] 10.222.0.40 () {60 vars in 1140 bytes} [Mon Jul 18 06:05:21 2022] POST /identity/v3/projects => generated 298 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:05:21.370279 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc53bd-8a11-4dfa-bd89-1196dd588bbe None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:21.398653 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:21.409808 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b7ae16-1627-4cf6-b8d9-eb8466f4ace8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:21.410662 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7ae16-1627-4cf6-b8d9-eb8466f4ace8 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:21.410955 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7ae16-1627-4cf6-b8d9-eb8466f4ace8 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:21.411240 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7ae16-1627-4cf6-b8d9-eb8466f4ace8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:21.412561 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b7ae16-1627-4cf6-b8d9-eb8466f4ace8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:21.414455 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b7ae16-1627-4cf6-b8d9-eb8466f4ace8 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:21.416271 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 28/56] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:05:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:05:21.423248 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcc3fad-d444-4b70-a823-e9ad0f060732 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:21.516524 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:21.555707 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcc3fad-d444-4b70-a823-e9ad0f060732 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:21.596908 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9102446b-53b7-4b6e-8e69-9c57c52ce9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:21.597593 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102446b-53b7-4b6e-8e69-9c57c52ce9ce None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:21.597891 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102446b-53b7-4b6e-8e69-9c57c52ce9ce None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:21.598405 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102446b-53b7-4b6e-8e69-9c57c52ce9ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:21.599020 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9102446b-53b7-4b6e-8e69-9c57c52ce9ce None admin] RBAC: Adding query filter params (role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc, scope.domain.id=default, user.id=8783051891e642af84b75a46b4d6b15e) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:21.599348 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9102446b-53b7-4b6e-8e69-9c57c52ce9ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:21.600862 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9102446b-53b7-4b6e-8e69-9c57c52ce9ce None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:21.608547 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 29/57] 10.222.0.40 () {56 vars in 1322 bytes} [Mon Jul 18 06:05:21 2022] GET /identity/v3/role_assignments?user.id=8783051891e642af84b75a46b4d6b15e&scope.domain.id=default&role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 530 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:23.610383 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be7d6b-b8b2-4c16-9834-13fdc96798de None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:23.611174 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be7d6b-b8b2-4c16-9834-13fdc96798de None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:23.611174 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be7d6b-b8b2-4c16-9834-13fdc96798de None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:23.612318 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 29/58] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:05:23.618059 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d5787-8154-4480-be77-a65f6415bbb5 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:23.618531 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d5787-8154-4480-be77-a65f6415bbb5 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:23.618658 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d5787-8154-4480-be77-a65f6415bbb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:23.635626 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-326d5787-8154-4480-be77-a65f6415bbb5 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:23.637406 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-326d5787-8154-4480-be77-a65f6415bbb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:23.712019 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 30/59] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 06:05:23.735900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be7d6b-b8b2-4c16-9834-13fdc96798de None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:23.849987 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be7d6b-b8b2-4c16-9834-13fdc96798de None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:23.886714 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afcde67-f0b6-492c-b91c-7c8dc0f625a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:23.888113 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afcde67-f0b6-492c-b91c-7c8dc0f625a5 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:23.888404 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afcde67-f0b6-492c-b91c-7c8dc0f625a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:23.888720 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afcde67-f0b6-492c-b91c-7c8dc0f625a5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:23.889961 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afcde67-f0b6-492c-b91c-7c8dc0f625a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:23.891435 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afcde67-f0b6-492c-b91c-7c8dc0f625a5 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:23.893144 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 30/60] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:05:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:23.898809 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d5787-8154-4480-be77-a65f6415bbb5 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:23.962668 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:24.003395 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d5787-8154-4480-be77-a65f6415bbb5 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:24.044782 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08632780-d88e-4a64-8bc1-8173449d8dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:24.045806 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08632780-d88e-4a64-8bc1-8173449d8dbf None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:24.046337 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08632780-d88e-4a64-8bc1-8173449d8dbf None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:24.046866 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08632780-d88e-4a64-8bc1-8173449d8dbf None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:24.048130 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08632780-d88e-4a64-8bc1-8173449d8dbf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:24.050892 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08632780-d88e-4a64-8bc1-8173449d8dbf None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:24.149484 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 31/61] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Jul 18 06:05:23 2022] POST /identity/v3/users => generated 275 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:05:27.398860 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0f158-a9bd-4d5a-be14-1c76761b466a None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:27.399470 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0f158-a9bd-4d5a-be14-1c76761b466a None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:27.399470 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0f158-a9bd-4d5a-be14-1c76761b466a None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:27.401125 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 31/62] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:05:27.409965 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3fd97-5cd6-4263-ac59-b0797872b723 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:27.410878 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3fd97-5cd6-4263-ac59-b0797872b723 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:27.411376 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3fd97-5cd6-4263-ac59-b0797872b723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:27.433440 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-38e3fd97-5cd6-4263-ac59-b0797872b723 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:27.435270 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-38e3fd97-5cd6-4263-ac59-b0797872b723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:27.521822 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 32/63] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:05:27.556642 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd0f158-a9bd-4d5a-be14-1c76761b466a None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:27.654855 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:27.726101 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd0f158-a9bd-4d5a-be14-1c76761b466a None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:27.765056 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18770321-970f-4e87-b06d-484d2fe55c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:27.766105 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18770321-970f-4e87-b06d-484d2fe55c3a None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:27.766563 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18770321-970f-4e87-b06d-484d2fe55c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:27.766993 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18770321-970f-4e87-b06d-484d2fe55c3a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:27.779030 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18770321-970f-4e87-b06d-484d2fe55c3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:05:27.787150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-18770321-970f-4e87-b06d-484d2fe55c3a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 06:05:27.787150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:05:27.787150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:05:27.787150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:05:27.787150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:05:27.787150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:05:27.787150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:05:27.787150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:05:27.787150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:05:27.787150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:05:27.787150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:05:27.787150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:05:27.787150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:05:27.787150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:05:27.787150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:05:27.787150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:05:27.788484 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:05:27.788484 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:05:27.788484 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:05:27.788484 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:05:27.788484 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:05:27.788484 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:05:27.788484 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:05:27.788484 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:05:27.788484 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:05:27.788484 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:05:27.788484 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:05:27.788484 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:05:27.788484 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:05:27.788484 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:05:27.788484 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:05:27.788484 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:05:27.789450 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:05:27.789450 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:05:27.789450 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:05:27.789450 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:05:27.789450 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:05:27.789450 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:05:27.789450 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:05:27.789450 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:05:27.789450 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:05:27.789450 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:05:27.789450 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 06:05:27.789450 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:05:27.791427 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 32/64] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:05:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:05:27.797891 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e3fd97-5cd6-4263-ac59-b0797872b723 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:27.807316 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60652d3-4ff5-479c-9917-1804fb7d1003 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:27.813243 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60652d3-4ff5-479c-9917-1804fb7d1003 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:27.813243 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60652d3-4ff5-479c-9917-1804fb7d1003 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:27.813243 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 33/65] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:05:27.816609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc564a26-175d-4a77-aeb9-38dbfaf7ba66 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:27.817065 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc564a26-175d-4a77-aeb9-38dbfaf7ba66 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:27.817505 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc564a26-175d-4a77-aeb9-38dbfaf7ba66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:27.847081 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-bc564a26-175d-4a77-aeb9-38dbfaf7ba66 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:27.849927 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-bc564a26-175d-4a77-aeb9-38dbfaf7ba66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:27.890257 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:28.012482 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 34/66] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 06:05:28.012482 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32a69f-c76d-4d02-9352-2e87e8098268 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:28.012482 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32a69f-c76d-4d02-9352-2e87e8098268 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:28.012482 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32a69f-c76d-4d02-9352-2e87e8098268 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:28.012482 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 35/67] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:27 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:05:28.012482 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447e093-c482-40a1-bc8d-1911a52fda01 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:28.012482 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447e093-c482-40a1-bc8d-1911a52fda01 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:28.013428 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447e093-c482-40a1-bc8d-1911a52fda01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:28.013502 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e3fd97-5cd6-4263-ac59-b0797872b723 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:28.013585 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-8447e093-c482-40a1-bc8d-1911a52fda01 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:28.013585 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8447e093-c482-40a1-bc8d-1911a52fda01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:28.026875 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6631c4a-3029-47ed-8e3d-17dcf58d12f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:28.042833 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6631c4a-3029-47ed-8e3d-17dcf58d12f9 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:28.042833 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6631c4a-3029-47ed-8e3d-17dcf58d12f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:28.042833 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6631c4a-3029-47ed-8e3d-17dcf58d12f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:28.042833 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6631c4a-3029-47ed-8e3d-17dcf58d12f9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:28.042833 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6631c4a-3029-47ed-8e3d-17dcf58d12f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:28.042833 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6631c4a-3029-47ed-8e3d-17dcf58d12f9 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:28.043794 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:28.043930 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 33/68] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 18 06:05:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:28.044640 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6631c4a-3029-47ed-8e3d-17dcf58d12f9 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:28.064460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 36/69] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:05:28.064460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8447e093-c482-40a1-bc8d-1911a52fda01 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:28.173305 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8447e093-c482-40a1-bc8d-1911a52fda01 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:28.173606 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6631c4a-3029-47ed-8e3d-17dcf58d12f9 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:28.185903 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:28.209787 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2ef55-b2e6-476e-95de-b722fb74f11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:28.210590 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2ef55-b2e6-476e-95de-b722fb74f11c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:28.210876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2ef55-b2e6-476e-95de-b722fb74f11c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:28.211157 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2ef55-b2e6-476e-95de-b722fb74f11c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:28.212836 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b7b46-2b0d-426d-9871-1d2f25e5c10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:28.213715 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b7b46-2b0d-426d-9871-1d2f25e5c10d None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:28.214000 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b7b46-2b0d-426d-9871-1d2f25e5c10d None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:28.214377 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b7b46-2b0d-426d-9871-1d2f25e5c10d None admin] PATH_INFO: `/v3/users/ab532fe0550b4d79a1bc56aff3a426f0` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:28.222252 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa2ef55-b2e6-476e-95de-b722fb74f11c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:05:28.229623 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15b7b46-2b0d-426d-9871-1d2f25e5c10d None admin] RBAC: Authorizing `identity:get_user(user_id=ab532fe0550b4d79a1bc56aff3a426f0)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:28.229882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-5aa2ef55-b2e6-476e-95de-b722fb74f11c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:05:28.229882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:05:28.229882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:05:28.229882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:05:28.229882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:05:28.229882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:05:28.229882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:05:28.229882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:05:28.229882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:05:28.229882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:05:28.229882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:05:28.229882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:05:28.229882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:05:28.229882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:05:28.229882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:05:28.229882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:05:28.230794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:05:28.230794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:05:28.230794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:05:28.230794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:05:28.230794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:05:28.230794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:05:28.230794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:05:28.230794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:05:28.230794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:05:28.230794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:05:28.230794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:05:28.230794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:05:28.230794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:05:28.230794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:05:28.230794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:05:28.230794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:05:28.231593 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:05:28.231593 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:05:28.231593 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:05:28.231593 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:05:28.231593 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:05:28.231593 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:05:28.231593 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:05:28.231593 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:05:28.231593 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:05:28.231593 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:05:28.231593 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:05:28.231593 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:05:28.231593 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 34/70] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 18 06:05:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:05:28.232366 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b4fac-92a8-4b07-b47f-aaefd88a9076 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:28.232847 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15b7b46-2b0d-426d-9871-1d2f25e5c10d None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:28.232951 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b4fac-92a8-4b07-b47f-aaefd88a9076 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:28.232951 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b4fac-92a8-4b07-b47f-aaefd88a9076 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:28.236460 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 35/71] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:05:28.236615 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 37/72] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:05:28 2022] GET /identity/v3/users/ab532fe0550b4d79a1bc56aff3a426f0 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:28.236699 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b4fac-92a8-4b07-b47f-aaefd88a9076 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:28.236775 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981273e7-7560-43d0-8148-bd33b7ffd44f None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:28.236775 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981273e7-7560-43d0-8148-bd33b7ffd44f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:28.236775 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981273e7-7560-43d0-8148-bd33b7ffd44f None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:28.241586 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 38/73] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:05:28.241586 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981273e7-7560-43d0-8148-bd33b7ffd44f None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:28.282975 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:28.345767 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b4fac-92a8-4b07-b47f-aaefd88a9076 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:28.347792 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981273e7-7560-43d0-8148-bd33b7ffd44f None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:28.380236 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0eed68-17e5-4806-b0b8-03bf3160d991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:28.381076 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0eed68-17e5-4806-b0b8-03bf3160d991 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:28.381363 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0eed68-17e5-4806-b0b8-03bf3160d991 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:28.381644 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0eed68-17e5-4806-b0b8-03bf3160d991 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:28.384217 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b6f231-6408-40e9-b8d8-80dcf5a4852f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:28.384923 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6f231-6408-40e9-b8d8-80dcf5a4852f None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:28.385208 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6f231-6408-40e9-b8d8-80dcf5a4852f None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:28.385485 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6f231-6408-40e9-b8d8-80dcf5a4852f None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:28.386244 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b6f231-6408-40e9-b8d8-80dcf5a4852f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:28.386554 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b6f231-6408-40e9-b8d8-80dcf5a4852f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:28.388636 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b6f231-6408-40e9-b8d8-80dcf5a4852f None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:28.392847 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0eed68-17e5-4806-b0b8-03bf3160d991 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:05:28.397292 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 39/74] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:05:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:28.398505 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba1eb9-9800-4ab5-9abb-a3f842c88765 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:28.398793 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba1eb9-9800-4ab5-9abb-a3f842c88765 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:28.399069 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba1eb9-9800-4ab5-9abb-a3f842c88765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:28.401112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-8d0eed68-17e5-4806-b0b8-03bf3160d991 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 06:05:28.401112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:05:28.401112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:05:28.401112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:05:28.401112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:05:28.401112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:05:28.401112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:05:28.401112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:05:28.401112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:05:28.401112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:05:28.401112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:05:28.401112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:05:28.401112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:05:28.401112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:05:28.401112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:05:28.401112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:05:28.401958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:05:28.401958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:05:28.401958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:05:28.401958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:05:28.401958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:05:28.401958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:05:28.401958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:05:28.401958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:05:28.401958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:05:28.401958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:05:28.401958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:05:28.401958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:05:28.401958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:05:28.401958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:05:28.401958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:05:28.401958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:05:28.402761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:05:28.402761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:05:28.402761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:05:28.402761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:05:28.402761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:05:28.402761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:05:28.402761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:05:28.402761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:05:28.402761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:05:28.402761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:05:28.402761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 06:05:28.402761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:05:28.402761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 36/75] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 18 06:05:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:05:28.404348 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ccae04-1e70-4fdd-9f52-94b7f9314559 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:28.404724 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ccae04-1e70-4fdd-9f52-94b7f9314559 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:28.404948 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ccae04-1e70-4fdd-9f52-94b7f9314559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:28.418299 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-29ba1eb9-9800-4ab5-9abb-a3f842c88765 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:28.419554 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-29ba1eb9-9800-4ab5-9abb-a3f842c88765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:28.421518 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-16ccae04-1e70-4fdd-9f52-94b7f9314559 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:28.422682 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-16ccae04-1e70-4fdd-9f52-94b7f9314559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:28.446972 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:28.487393 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 40/76] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:05:28.488247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ba1eb9-9800-4ab5-9abb-a3f842c88765 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:28.501248 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 37/77] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:05:28.501248 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ccae04-1e70-4fdd-9f52-94b7f9314559 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:28.519431 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:28.590263 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ba1eb9-9800-4ab5-9abb-a3f842c88765 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:28.612628 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ccae04-1e70-4fdd-9f52-94b7f9314559 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:28.624637 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8016052-4e84-4d67-b615-9257228112e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:28.625297 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8016052-4e84-4d67-b615-9257228112e9 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:28.625593 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8016052-4e84-4d67-b615-9257228112e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:28.625879 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8016052-4e84-4d67-b615-9257228112e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:28.647763 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f726201-748d-4c5a-965b-e3990943a91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:28.648676 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f726201-748d-4c5a-965b-e3990943a91f None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:28.648972 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f726201-748d-4c5a-965b-e3990943a91f None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:28.649254 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f726201-748d-4c5a-965b-e3990943a91f None admin] PATH_INFO: `/v3/users/8783051891e642af84b75a46b4d6b15e` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:28.650454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f726201-748d-4c5a-965b-e3990943a91f None admin] RBAC: Authorizing `identity:get_user(user_id=8783051891e642af84b75a46b4d6b15e)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:28.651954 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f726201-748d-4c5a-965b-e3990943a91f None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:28.653919 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 38/78] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:05:28 2022] GET /identity/v3/users/8783051891e642af84b75a46b4d6b15e => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:28.654718 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f726201-748d-4c5a-965b-e3990943a91f None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:28.684639 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:28.698299 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8016052-4e84-4d67-b615-9257228112e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:28.699894 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8016052-4e84-4d67-b615-9257228112e9 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:28.728609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:28.759051 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f726201-748d-4c5a-965b-e3990943a91f None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:28.796747 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c7ac5-4ea3-48e6-914b-70a557543018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:28.797483 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c7ac5-4ea3-48e6-914b-70a557543018 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:28.797775 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c7ac5-4ea3-48e6-914b-70a557543018 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:28.798096 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c7ac5-4ea3-48e6-914b-70a557543018 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:28.798843 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c7ac5-4ea3-48e6-914b-70a557543018 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:28.799138 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c7ac5-4ea3-48e6-914b-70a557543018 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:28.800583 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c7ac5-4ea3-48e6-914b-70a557543018 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:28.802567 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 41/79] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:05:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 06:05:28.803198 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8016052-4e84-4d67-b615-9257228112e9 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:28.809897 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 39/80] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:05:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:28.810859 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c7ac5-4ea3-48e6-914b-70a557543018 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:28.903142 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:28.910071 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8016052-4e84-4d67-b615-9257228112e9 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:28.912865 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c7ac5-4ea3-48e6-914b-70a557543018 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:28.942782 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:28.948112 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d44cc-ff86-444a-a234-f3280ceb5f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:28.948940 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d44cc-ff86-444a-a234-f3280ceb5f16 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:28.949277 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d44cc-ff86-444a-a234-f3280ceb5f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:28.949609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d44cc-ff86-444a-a234-f3280ceb5f16 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:28.950094 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f329c7b-81ca-49e5-851d-130f3645f77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:28.951090 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f329c7b-81ca-49e5-851d-130f3645f77d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:28.951377 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f329c7b-81ca-49e5-851d-130f3645f77d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:28.951655 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f329c7b-81ca-49e5-851d-130f3645f77d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:28.952890 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f329c7b-81ca-49e5-851d-130f3645f77d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:28.954355 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f329c7b-81ca-49e5-851d-130f3645f77d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:28.956250 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 40/81] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:05:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:28.956980 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f329c7b-81ca-49e5-851d-130f3645f77d None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:28.964565 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d44cc-ff86-444a-a234-f3280ceb5f16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:05:28.973769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-333d44cc-ff86-444a-a234-f3280ceb5f16 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 06:05:28.973769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:05:28.973769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:05:28.973769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:05:28.973769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:05:28.973769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:05:28.973769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:05:28.973769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:05:28.973769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:05:28.973769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:05:28.973769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:05:28.973769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:05:28.973769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:05:28.973769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:05:28.973769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:05:28.973769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:05:28.974892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:05:28.974892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:05:28.974892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:05:28.974892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:05:28.974892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:05:28.974892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:05:28.974892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:05:28.974892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:05:28.974892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:05:28.974892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:05:28.974892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:05:28.974892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:05:28.974892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:05:28.974892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:05:28.974892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:05:28.974892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:05:28.975902 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:05:28.975902 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:05:28.975902 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:05:28.975902 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:05:28.975902 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:05:28.975902 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:05:28.975902 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:05:28.975902 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:05:28.975902 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:05:28.975902 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:05:28.975902 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 06:05:28.975902 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:05:28.979099 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 42/82] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:05:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:05:28.980033 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d44cc-ff86-444a-a234-f3280ceb5f16 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:29.066774 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f329c7b-81ca-49e5-851d-130f3645f77d None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:29.086588 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d44cc-ff86-444a-a234-f3280ceb5f16 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:29.104191 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8984ae5c-696c-4fc7-ae4b-39903a92f79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:29.104978 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8984ae5c-696c-4fc7-ae4b-39903a92f79b None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:29.105277 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8984ae5c-696c-4fc7-ae4b-39903a92f79b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:29.105646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8984ae5c-696c-4fc7-ae4b-39903a92f79b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:29.123720 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f114c505-a81f-4dfa-ba43-1d240ec57448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:29.124782 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114c505-a81f-4dfa-ba43-1d240ec57448 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:29.125080 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114c505-a81f-4dfa-ba43-1d240ec57448 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:29.125366 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114c505-a81f-4dfa-ba43-1d240ec57448 None admin] PATH_INFO: `/v3/projects/4620d11733d2401eb2122d406221cf0d` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:29.126608 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f114c505-a81f-4dfa-ba43-1d240ec57448 None admin] RBAC: Authorizing `identity:get_project(project_id=4620d11733d2401eb2122d406221cf0d)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:29.128083 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f114c505-a81f-4dfa-ba43-1d240ec57448 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:29.129847 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 43/83] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:05:28 2022] GET /identity/v3/projects/4620d11733d2401eb2122d406221cf0d => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:29.130631 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f114c505-a81f-4dfa-ba43-1d240ec57448 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:29.140841 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:29.184568 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8984ae5c-696c-4fc7-ae4b-39903a92f79b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:29.186030 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8984ae5c-696c-4fc7-ae4b-39903a92f79b None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:29.191477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:29.236085 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f114c505-a81f-4dfa-ba43-1d240ec57448 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:29.287451 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ace37-e420-4b7d-83f2-9aa0cc2792e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:29.288656 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ace37-e420-4b7d-83f2-9aa0cc2792e5 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:29.288948 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ace37-e420-4b7d-83f2-9aa0cc2792e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:29.289236 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ace37-e420-4b7d-83f2-9aa0cc2792e5 None admin] PATH_INFO: `/v3/users/ab532fe0550b4d79a1bc56aff3a426f0` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:29.290525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ace37-e420-4b7d-83f2-9aa0cc2792e5 None admin] RBAC: Authorizing `identity:get_user(user_id=ab532fe0550b4d79a1bc56aff3a426f0)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:29.291980 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ace37-e420-4b7d-83f2-9aa0cc2792e5 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:29.292859 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 41/84] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:05:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 06:05:29.294001 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 44/85] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:05:29 2022] GET /identity/v3/users/ab532fe0550b4d79a1bc56aff3a426f0 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:29.294106 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8984ae5c-696c-4fc7-ae4b-39903a92f79b None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:29.297489 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ace37-e420-4b7d-83f2-9aa0cc2792e5 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:29.356879 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:29.401200 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8984ae5c-696c-4fc7-ae4b-39903a92f79b None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:29.409119 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ace37-e420-4b7d-83f2-9aa0cc2792e5 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:29.448541 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:29.451749 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b5f52-c3f1-4ef6-acf0-53b5bc2e31fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:29.452498 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b5f52-c3f1-4ef6-acf0-53b5bc2e31fd None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:29.452800 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b5f52-c3f1-4ef6-acf0-53b5bc2e31fd None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:29.453085 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b5f52-c3f1-4ef6-acf0-53b5bc2e31fd None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:29.453802 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409b5f52-c3f1-4ef6-acf0-53b5bc2e31fd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:29.455275 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409b5f52-c3f1-4ef6-acf0-53b5bc2e31fd None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:29.458580 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4669b051-b14d-4518-a26b-28346ac2ecec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:29.482110 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669b051-b14d-4518-a26b-28346ac2ecec None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:29.482110 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669b051-b14d-4518-a26b-28346ac2ecec None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:29.482110 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669b051-b14d-4518-a26b-28346ac2ecec None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:29.482110 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4669b051-b14d-4518-a26b-28346ac2ecec None admin] RBAC: Adding query filter params (name=member) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:29.482110 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4669b051-b14d-4518-a26b-28346ac2ecec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:29.482110 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4669b051-b14d-4518-a26b-28346ac2ecec None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:29.482613 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 45/86] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 18 06:05:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:29.482613 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4669b051-b14d-4518-a26b-28346ac2ecec None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:29.575295 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4669b051-b14d-4518-a26b-28346ac2ecec None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:29.581920 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 42/87] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Jul 18 06:05:29 2022] POST /identity/v3/users => generated 289 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:05:29.597513 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b5f52-c3f1-4ef6-acf0-53b5bc2e31fd None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:29.618305 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec60bb5-3cef-40a3-a391-d6d4caa51a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:29.619040 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec60bb5-3cef-40a3-a391-d6d4caa51a30 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:29.619370 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec60bb5-3cef-40a3-a391-d6d4caa51a30 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:29.619659 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec60bb5-3cef-40a3-a391-d6d4caa51a30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:29.620682 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec60bb5-3cef-40a3-a391-d6d4caa51a30 None admin] RBAC: Adding query filter params (role.id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20, scope.project.id=4620d11733d2401eb2122d406221cf0d, user.id=ab532fe0550b4d79a1bc56aff3a426f0) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:29.621883 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec60bb5-3cef-40a3-a391-d6d4caa51a30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:29.623956 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec60bb5-3cef-40a3-a391-d6d4caa51a30 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:29.644449 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 46/88] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:05:29 2022] GET /identity/v3/role_assignments?user.id=ab532fe0550b4d79a1bc56aff3a426f0&scope.project.id=4620d11733d2401eb2122d406221cf0d&role.id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:29.644449 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec60bb5-3cef-40a3-a391-d6d4caa51a30 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:29.742339 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:29.758834 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b5f52-c3f1-4ef6-acf0-53b5bc2e31fd None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:29.781393 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:29.811216 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f05b9b8-c1b4-4dba-a291-9c347228440b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:29.812758 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05b9b8-c1b4-4dba-a291-9c347228440b None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:29.815983 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05b9b8-c1b4-4dba-a291-9c347228440b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:29.815983 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05b9b8-c1b4-4dba-a291-9c347228440b None admin] PATH_INFO: `/v3/projects/4620d11733d2401eb2122d406221cf0d` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:29.815983 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f05b9b8-c1b4-4dba-a291-9c347228440b None admin] RBAC: Authorizing `identity:get_project(project_id=4620d11733d2401eb2122d406221cf0d)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:29.816211 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec60bb5-3cef-40a3-a391-d6d4caa51a30 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:29.816616 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f05b9b8-c1b4-4dba-a291-9c347228440b None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:29.818840 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 43/89] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:05:29 2022] GET /identity/v3/projects/4620d11733d2401eb2122d406221cf0d => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:29.819668 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f05b9b8-c1b4-4dba-a291-9c347228440b None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:29.927999 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff06b7fa-c1c0-42c8-b35b-d5fcfe496197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:29.928749 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff06b7fa-c1c0-42c8-b35b-d5fcfe496197 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:29.929068 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff06b7fa-c1c0-42c8-b35b-d5fcfe496197 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:29.929328 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff06b7fa-c1c0-42c8-b35b-d5fcfe496197 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:30.017746 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f05b9b8-c1b4-4dba-a291-9c347228440b None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:30.023310 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff06b7fa-c1c0-42c8-b35b-d5fcfe496197 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:30.025195 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff06b7fa-c1c0-42c8-b35b-d5fcfe496197 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:30.065573 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf706ff-6ea7-4afc-823c-82d3a0fb4c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:30.066479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf706ff-6ea7-4afc-823c-82d3a0fb4c7f None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:30.066773 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf706ff-6ea7-4afc-823c-82d3a0fb4c7f None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:30.067062 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf706ff-6ea7-4afc-823c-82d3a0fb4c7f None admin] PATH_INFO: `/v3/users/ab532fe0550b4d79a1bc56aff3a426f0` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:30.068308 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf706ff-6ea7-4afc-823c-82d3a0fb4c7f None admin] RBAC: Authorizing `identity:get_user(user_id=ab532fe0550b4d79a1bc56aff3a426f0)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:30.069854 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf706ff-6ea7-4afc-823c-82d3a0fb4c7f None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:30.071591 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 44/90] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:05:29 2022] GET /identity/v3/users/ab532fe0550b4d79a1bc56aff3a426f0 => generated 275 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:30.072391 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf706ff-6ea7-4afc-823c-82d3a0fb4c7f None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:30.109093 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:30.170056 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:30.234315 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf706ff-6ea7-4afc-823c-82d3a0fb4c7f None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:30.271157 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 47/91] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:05:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 636 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 06:05:30.271998 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff06b7fa-c1c0-42c8-b35b-d5fcfe496197 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:30.273571 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e8a85f-b9f5-4748-9050-9719d9e56402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:30.274470 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8a85f-b9f5-4748-9050-9719d9e56402 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:30.274959 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8a85f-b9f5-4748-9050-9719d9e56402 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:30.275614 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8a85f-b9f5-4748-9050-9719d9e56402 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:30.276703 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e8a85f-b9f5-4748-9050-9719d9e56402 None admin] RBAC: Adding query filter params (role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc, scope.project.id=4620d11733d2401eb2122d406221cf0d, user.id=8783051891e642af84b75a46b4d6b15e) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:30.277225 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e8a85f-b9f5-4748-9050-9719d9e56402 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:30.279624 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e8a85f-b9f5-4748-9050-9719d9e56402 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:30.286220 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 45/92] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:05:30 2022] GET /identity/v3/role_assignments?user.id=8783051891e642af84b75a46b4d6b15e&scope.project.id=4620d11733d2401eb2122d406221cf0d&role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:30.287596 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e8a85f-b9f5-4748-9050-9719d9e56402 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:30.415475 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e8a85f-b9f5-4748-9050-9719d9e56402 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:30.431976 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:30.455219 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d4d736-d95d-471b-b70a-07aabea629fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:30.456973 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4d736-d95d-471b-b70a-07aabea629fc None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:30.457488 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4d736-d95d-471b-b70a-07aabea629fc None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:30.457985 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4d736-d95d-471b-b70a-07aabea629fc None admin] PATH_INFO: `/v3/projects/4620d11733d2401eb2122d406221cf0d` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:30.459541 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d4d736-d95d-471b-b70a-07aabea629fc None admin] RBAC: Authorizing `identity:get_project(project_id=4620d11733d2401eb2122d406221cf0d)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:30.461486 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d4d736-d95d-471b-b70a-07aabea629fc None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:30.466106 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 46/93] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:05:30 2022] GET /identity/v3/projects/4620d11733d2401eb2122d406221cf0d => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:30.466301 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff06b7fa-c1c0-42c8-b35b-d5fcfe496197 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:30.473912 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d4d736-d95d-471b-b70a-07aabea629fc None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:30.509385 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2971994-21f8-442a-87b8-1e44b6588bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:30.509799 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2971994-21f8-442a-87b8-1e44b6588bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:30.510098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2971994-21f8-442a-87b8-1e44b6588bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:30.510396 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2971994-21f8-442a-87b8-1e44b6588bc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:30.552937 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:30.662790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2971994-21f8-442a-87b8-1e44b6588bc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:30.664659 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2971994-21f8-442a-87b8-1e44b6588bc3 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:30.672467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d4d736-d95d-471b-b70a-07aabea629fc None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:30.723398 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef8f859-fa0e-4759-b883-0ee773c5bc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:30.724513 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8f859-fa0e-4759-b883-0ee773c5bc30 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:30.725004 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8f859-fa0e-4759-b883-0ee773c5bc30 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:30.725493 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8f859-fa0e-4759-b883-0ee773c5bc30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:30.726897 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef8f859-fa0e-4759-b883-0ee773c5bc30 None admin] RBAC: Adding query filter params (role.id=04868303225144c389068779288f3a42, scope.project.id=4620d11733d2401eb2122d406221cf0d, user.id=ab532fe0550b4d79a1bc56aff3a426f0) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:30.727735 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef8f859-fa0e-4759-b883-0ee773c5bc30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:30.729437 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef8f859-fa0e-4759-b883-0ee773c5bc30 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:30.737342 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 47/94] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:05:30 2022] GET /identity/v3/role_assignments?user.id=ab532fe0550b4d79a1bc56aff3a426f0&scope.project.id=4620d11733d2401eb2122d406221cf0d&role.id=04868303225144c389068779288f3a42 => generated 259 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:30.878545 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 48/95] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:05:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 06:05:30.884403 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef8f859-fa0e-4759-b883-0ee773c5bc30 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:30.976445 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:31.032761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef8f859-fa0e-4759-b883-0ee773c5bc30 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:31.070323 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e509f5-25b2-45cc-b487-5d729cbe5b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:31.071562 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e509f5-25b2-45cc-b487-5d729cbe5b54 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:31.072047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e509f5-25b2-45cc-b487-5d729cbe5b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:31.072622 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e509f5-25b2-45cc-b487-5d729cbe5b54 None admin] PATH_INFO: `/v3/projects/1a5e942e8e934867bf5c202585b4fa2b` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:31.074058 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e509f5-25b2-45cc-b487-5d729cbe5b54 None admin] RBAC: Authorizing `identity:get_project(project_id=1a5e942e8e934867bf5c202585b4fa2b)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:31.076599 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e509f5-25b2-45cc-b487-5d729cbe5b54 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:31.078651 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 48/96] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:05:30 2022] GET /identity/v3/projects/1a5e942e8e934867bf5c202585b4fa2b => generated 312 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:05:31.084695 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2971994-21f8-442a-87b8-1e44b6588bc3 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:31.167672 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:31.229411 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2971994-21f8-442a-87b8-1e44b6588bc3 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:31.281233 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc2bd22-8274-4667-a183-b37b8837bb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:31.281927 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2bd22-8274-4667-a183-b37b8837bb47 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:31.282223 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2bd22-8274-4667-a183-b37b8837bb47 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:31.282514 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2bd22-8274-4667-a183-b37b8837bb47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:31.283415 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc2bd22-8274-4667-a183-b37b8837bb47 None admin] RBAC: Adding query filter params (role.id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20, scope.project.id=1a5e942e8e934867bf5c202585b4fa2b, user.id=ab532fe0550b4d79a1bc56aff3a426f0) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:31.283739 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc2bd22-8274-4667-a183-b37b8837bb47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:31.285242 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc2bd22-8274-4667-a183-b37b8837bb47 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:31.292264 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 49/97] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:05:31 2022] GET /identity/v3/role_assignments?user.id=ab532fe0550b4d79a1bc56aff3a426f0&scope.project.id=1a5e942e8e934867bf5c202585b4fa2b&role.id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:32.530798 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85561f-5f7b-4740-b696-f2421f0b9010 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:32.531813 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85561f-5f7b-4740-b696-f2421f0b9010 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:32.532297 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85561f-5f7b-4740-b696-f2421f0b9010 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:32.533928 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 49/98] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:05:32.539647 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1856fc-e799-4e3a-a903-b729cdde04f9 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:32.540618 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1856fc-e799-4e3a-a903-b729cdde04f9 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:32.541105 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1856fc-e799-4e3a-a903-b729cdde04f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:32.565506 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-bb1856fc-e799-4e3a-a903-b729cdde04f9 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:32.567074 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-bb1856fc-e799-4e3a-a903-b729cdde04f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:32.668101 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 50/99] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:05:32.710625 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd85561f-5f7b-4740-b696-f2421f0b9010 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:32.858411 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd85561f-5f7b-4740-b696-f2421f0b9010 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:32.877528 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:32.904919 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e54ba16-dcab-4ffa-a2db-7b94e37f8030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:32.905735 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54ba16-dcab-4ffa-a2db-7b94e37f8030 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:32.906023 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54ba16-dcab-4ffa-a2db-7b94e37f8030 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:32.906306 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54ba16-dcab-4ffa-a2db-7b94e37f8030 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:32.924606 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e54ba16-dcab-4ffa-a2db-7b94e37f8030 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:05:32.937576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-2e54ba16-dcab-4ffa-a2db-7b94e37f8030 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 06:05:32.937576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:05:32.937576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:05:32.937576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:05:32.937576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:05:32.937576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:05:32.937576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:05:32.937576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:05:32.937576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:05:32.937576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:05:32.937576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:05:32.937576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:05:32.937576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:05:32.937576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:05:32.937576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:05:32.937576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:05:32.938620 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:05:32.938620 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:05:32.938620 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:05:32.938620 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:05:32.938620 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:05:32.938620 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:05:32.938620 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:05:32.938620 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:05:32.938620 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:05:32.938620 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:05:32.938620 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:05:32.938620 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:05:32.938620 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:05:32.938620 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:05:32.938620 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:05:32.938620 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:05:32.939392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:05:32.939392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:05:32.939392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:05:32.939392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:05:32.939392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:05:32.939392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:05:32.939392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:05:32.939392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:05:32.939392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:05:32.939392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:05:32.939392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 06:05:32.939392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:05:32.939392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 50/100] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:05:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:05:32.946264 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1856fc-e799-4e3a-a903-b729cdde04f9 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:33.015967 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:33.085987 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1856fc-e799-4e3a-a903-b729cdde04f9 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:33.139295 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b85db7-e75a-4b76-b3fe-d18d4f6c591f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:33.139989 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b85db7-e75a-4b76-b3fe-d18d4f6c591f None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:33.140284 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b85db7-e75a-4b76-b3fe-d18d4f6c591f None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:33.140603 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b85db7-e75a-4b76-b3fe-d18d4f6c591f None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:33.141387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b85db7-e75a-4b76-b3fe-d18d4f6c591f None admin] RBAC: Adding query filter params (name=member) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:33.141705 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b85db7-e75a-4b76-b3fe-d18d4f6c591f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:33.143119 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b85db7-e75a-4b76-b3fe-d18d4f6c591f None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:33.157214 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 51/101] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 18 06:05:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:33.162797 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e54ba16-dcab-4ffa-a2db-7b94e37f8030 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:33.218390 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1c056-7a99-4c8b-845d-4969e625738c None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:33.219043 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1c056-7a99-4c8b-845d-4969e625738c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:33.219531 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1c056-7a99-4c8b-845d-4969e625738c None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:33.221196 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 52/102] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:05:33.226806 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e2fe7-f013-47cb-940e-ad7b892e723c None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:33.227396 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e2fe7-f013-47cb-940e-ad7b892e723c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:33.227879 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e2fe7-f013-47cb-940e-ad7b892e723c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:33.247080 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-657e2fe7-f013-47cb-940e-ad7b892e723c None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:33.248812 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-657e2fe7-f013-47cb-940e-ad7b892e723c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:33.371839 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 53/103] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 06:05:33.380437 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d80354-11c9-41c1-9b54-7090ff55d48b None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:33.381314 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d80354-11c9-41c1-9b54-7090ff55d48b None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:33.381834 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d80354-11c9-41c1-9b54-7090ff55d48b None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:33.383525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 54/104] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:33 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:05:33.396169 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da1de6-cf87-4759-970d-104d5623e44a None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:33.396743 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da1de6-cf87-4759-970d-104d5623e44a None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:33.397227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da1de6-cf87-4759-970d-104d5623e44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:33.409506 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e54ba16-dcab-4ffa-a2db-7b94e37f8030 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:33.429697 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-21da1de6-cf87-4759-970d-104d5623e44a None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:33.431397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-21da1de6-cf87-4759-970d-104d5623e44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:33.489182 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:33.491051 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cbc50d-d792-4a5e-8d9a-d4ad870558d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:33.493203 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbc50d-d792-4a5e-8d9a-d4ad870558d5 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:33.494336 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbc50d-d792-4a5e-8d9a-d4ad870558d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:33.495322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbc50d-d792-4a5e-8d9a-d4ad870558d5 None admin] PATH_INFO: `/v3/users/ab532fe0550b4d79a1bc56aff3a426f0` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:33.498518 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cbc50d-d792-4a5e-8d9a-d4ad870558d5 None admin] RBAC: Authorizing `identity:get_user(user_id=ab532fe0550b4d79a1bc56aff3a426f0)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:33.500393 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cbc50d-d792-4a5e-8d9a-d4ad870558d5 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:33.506871 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 51/105] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:05:33 2022] GET /identity/v3/users/ab532fe0550b4d79a1bc56aff3a426f0 => generated 275 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:33.506871 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cbc50d-d792-4a5e-8d9a-d4ad870558d5 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:33.554368 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:33.557857 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 55/106] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:05:33.560239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6953b-4905-49dc-8d66-93c0c3aee61b None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:33.560979 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6953b-4905-49dc-8d66-93c0c3aee61b None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:33.561475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6953b-4905-49dc-8d66-93c0c3aee61b None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:33.563487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 56/107] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:05:33.568834 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6953b-4905-49dc-8d66-93c0c3aee61b None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:33.647154 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cbc50d-d792-4a5e-8d9a-d4ad870558d5 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:33.703696 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a94839e-0a39-4274-915a-da645cedf728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:33.704912 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94839e-0a39-4274-915a-da645cedf728 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:33.705265 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94839e-0a39-4274-915a-da645cedf728 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:33.705631 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94839e-0a39-4274-915a-da645cedf728 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:33.719300 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6953b-4905-49dc-8d66-93c0c3aee61b None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:33.722169 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a94839e-0a39-4274-915a-da645cedf728 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:05:33.732733 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-6a94839e-0a39-4274-915a-da645cedf728 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:05:33.732733 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:05:33.732733 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:05:33.732733 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:05:33.732733 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:05:33.732733 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:05:33.732733 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:05:33.732733 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:05:33.732733 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:05:33.732733 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:05:33.732733 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:05:33.732733 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:05:33.732733 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:05:33.732733 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:05:33.732733 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:05:33.732733 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:05:33.733882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:05:33.733882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:05:33.733882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:05:33.733882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:05:33.733882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:05:33.733882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:05:33.733882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:05:33.733882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:05:33.733882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:05:33.733882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:05:33.733882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:05:33.733882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:05:33.733882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:05:33.733882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:05:33.733882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:05:33.733882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:05:33.734860 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:05:33.734860 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:05:33.734860 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:05:33.734860 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:05:33.734860 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:05:33.734860 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:05:33.734860 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:05:33.734860 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:05:33.734860 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:05:33.734860 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:05:33.734860 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:05:33.734860 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:05:33.735900 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 52/108] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 18 06:05:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:05:33.737347 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7080f-9e19-49d6-9b11-f527d64507bb None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:33.737820 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7080f-9e19-49d6-9b11-f527d64507bb None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:33.738243 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7080f-9e19-49d6-9b11-f527d64507bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:33.761262 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-46b7080f-9e19-49d6-9b11-f527d64507bb None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:33.763564 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-46b7080f-9e19-49d6-9b11-f527d64507bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:33.771622 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc849038-e630-41c3-ad25-e93a7b4f40b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:33.773065 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc849038-e630-41c3-ad25-e93a7b4f40b5 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:33.773568 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc849038-e630-41c3-ad25-e93a7b4f40b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:33.774553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc849038-e630-41c3-ad25-e93a7b4f40b5 None admin] PATH_INFO: `/v3/projects/4620d11733d2401eb2122d406221cf0d` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:33.776373 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc849038-e630-41c3-ad25-e93a7b4f40b5 None admin] RBAC: Authorizing `identity:get_project(project_id=4620d11733d2401eb2122d406221cf0d)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:33.779093 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc849038-e630-41c3-ad25-e93a7b4f40b5 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:33.781624 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 57/109] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:05:33 2022] GET /identity/v3/projects/4620d11733d2401eb2122d406221cf0d => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:33.783108 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc849038-e630-41c3-ad25-e93a7b4f40b5 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:33.833418 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:33.855337 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 53/110] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:05:33.856346 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7080f-9e19-49d6-9b11-f527d64507bb None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:33.900905 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:33.919238 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc849038-e630-41c3-ad25-e93a7b4f40b5 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:33.973572 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04da0eb-d152-4485-893c-c36321f7fff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:33.975050 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04da0eb-d152-4485-893c-c36321f7fff9 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:33.975799 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04da0eb-d152-4485-893c-c36321f7fff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:33.976307 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04da0eb-d152-4485-893c-c36321f7fff9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:33.992791 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04da0eb-d152-4485-893c-c36321f7fff9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:05:34.000895 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7080f-9e19-49d6-9b11-f527d64507bb None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:34.005246 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-b04da0eb-d152-4485-893c-c36321f7fff9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 06:05:34.005246 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:05:34.005246 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:05:34.005246 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:05:34.005246 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:05:34.005246 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:05:34.005246 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:05:34.005246 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:05:34.005246 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:05:34.005246 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:05:34.005246 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:05:34.005246 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:05:34.005246 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:05:34.005246 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:05:34.005246 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:05:34.005246 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:05:34.006126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:05:34.006126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:05:34.006126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:05:34.006126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:05:34.006126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:05:34.006126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:05:34.006126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:05:34.006126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:05:34.006126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:05:34.006126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:05:34.006126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:05:34.006126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:05:34.006126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:05:34.006126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:05:34.006126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:05:34.006126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:05:34.006910 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:05:34.006910 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:05:34.006910 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:05:34.006910 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:05:34.006910 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:05:34.006910 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:05:34.006910 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:05:34.006910 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:05:34.006910 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:05:34.006910 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:05:34.006910 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 06:05:34.006910 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:05:34.010005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 58/111] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:05:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:05:34.011561 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04da0eb-d152-4485-893c-c36321f7fff9 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:34.039576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6016b7fb-e190-4ebc-9ea0-7ffd69cb359d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:34.040259 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016b7fb-e190-4ebc-9ea0-7ffd69cb359d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:34.040807 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016b7fb-e190-4ebc-9ea0-7ffd69cb359d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:34.040912 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016b7fb-e190-4ebc-9ea0-7ffd69cb359d None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:34.041648 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6016b7fb-e190-4ebc-9ea0-7ffd69cb359d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:34.041963 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6016b7fb-e190-4ebc-9ea0-7ffd69cb359d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:34.044072 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6016b7fb-e190-4ebc-9ea0-7ffd69cb359d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:34.053890 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 54/112] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:05:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:34.054860 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6016b7fb-e190-4ebc-9ea0-7ffd69cb359d None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:34.135553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:34.141664 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04da0eb-d152-4485-893c-c36321f7fff9 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:34.157589 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:34.165100 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6016b7fb-e190-4ebc-9ea0-7ffd69cb359d None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:34.179831 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cc7256-5344-4a46-bc2e-5b76ff249a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:34.181194 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cc7256-5344-4a46-bc2e-5b76ff249a61 None admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:34.181738 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cc7256-5344-4a46-bc2e-5b76ff249a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:34.182202 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cc7256-5344-4a46-bc2e-5b76ff249a61 None admin] PATH_INFO: `/v3/projects/4620d11733d2401eb2122d406221cf0d/users/ab532fe0550b4d79a1bc56aff3a426f0/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:34.185264 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cc7256-5344-4a46-bc2e-5b76ff249a61 None admin] RBAC: Authorizing `identity:create_grant(project_id=4620d11733d2401eb2122d406221cf0d, user_id=ab532fe0550b4d79a1bc56aff3a426f0, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:34.186996 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cc7256-5344-4a46-bc2e-5b76ff249a61 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:34.202980 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85fe266-0396-462d-a74e-35a3568cd955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:34.203785 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fe266-0396-462d-a74e-35a3568cd955 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:34.204074 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fe266-0396-462d-a74e-35a3568cd955 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:34.204354 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fe266-0396-462d-a74e-35a3568cd955 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:34.205624 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85fe266-0396-462d-a74e-35a3568cd955 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:34.207090 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85fe266-0396-462d-a74e-35a3568cd955 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:34.208942 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 55/113] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:05:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:34.209663 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85fe266-0396-462d-a74e-35a3568cd955 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:34.279708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 59/114] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 18 06:05:34 2022] PUT /identity/v3/projects/4620d11733d2401eb2122d406221cf0d/users/ab532fe0550b4d79a1bc56aff3a426f0/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:05:34.280605 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cc7256-5344-4a46-bc2e-5b76ff249a61 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:34.333628 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85fe266-0396-462d-a74e-35a3568cd955 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:34.374298 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344c3d82-b5a4-4716-9e38-1b13ae9eab9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:34.375004 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c3d82-b5a4-4716-9e38-1b13ae9eab9c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:34.375347 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c3d82-b5a4-4716-9e38-1b13ae9eab9c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:34.375688 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c3d82-b5a4-4716-9e38-1b13ae9eab9c None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:34.376503 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344c3d82-b5a4-4716-9e38-1b13ae9eab9c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:34.376822 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344c3d82-b5a4-4716-9e38-1b13ae9eab9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:34.378299 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344c3d82-b5a4-4716-9e38-1b13ae9eab9c None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:34.387415 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 56/115] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 18 06:05:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:34.393678 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9957f-0579-4a9e-921f-c71b9bb48a26 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:34.393973 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9957f-0579-4a9e-921f-c71b9bb48a26 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:34.394264 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9957f-0579-4a9e-921f-c71b9bb48a26 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:34.395693 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 57/116] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:05:34.396460 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb9957f-0579-4a9e-921f-c71b9bb48a26 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:34.405226 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cc7256-5344-4a46-bc2e-5b76ff249a61 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:34.428104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:34.448126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8496c1dc-dfd8-4c07-8631-c540f0cf4947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:34.449497 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496c1dc-dfd8-4c07-8631-c540f0cf4947 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:34.449497 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496c1dc-dfd8-4c07-8631-c540f0cf4947 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:34.449680 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496c1dc-dfd8-4c07-8631-c540f0cf4947 None admin] PATH_INFO: `/v3/users/8783051891e642af84b75a46b4d6b15e` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:34.450876 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8496c1dc-dfd8-4c07-8631-c540f0cf4947 None admin] RBAC: Authorizing `identity:get_user(user_id=8783051891e642af84b75a46b4d6b15e)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:34.452347 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8496c1dc-dfd8-4c07-8631-c540f0cf4947 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:34.454092 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 60/117] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:05:34 2022] GET /identity/v3/users/8783051891e642af84b75a46b4d6b15e => generated 247 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:34.455297 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088f0e6-cb4a-4ac6-a10a-84ab49ed5ae2 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:34.455591 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088f0e6-cb4a-4ac6-a10a-84ab49ed5ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:34.455874 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088f0e6-cb4a-4ac6-a10a-84ab49ed5ae2 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:34.457306 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 61/118] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:05:34.457834 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2088f0e6-cb4a-4ac6-a10a-84ab49ed5ae2 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:34.487972 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:34.519207 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb9957f-0579-4a9e-921f-c71b9bb48a26 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:34.556297 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47ec49-218e-4523-b91a-927279892954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:34.557019 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47ec49-218e-4523-b91a-927279892954 None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:34.557312 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47ec49-218e-4523-b91a-927279892954 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:34.557599 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47ec49-218e-4523-b91a-927279892954 None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:34.558328 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f47ec49-218e-4523-b91a-927279892954 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:34.560369 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f47ec49-218e-4523-b91a-927279892954 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:34.579199 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2088f0e6-cb4a-4ac6-a10a-84ab49ed5ae2 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:34.613340 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb882742-96cb-442a-9acd-ac72d613faa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:34.614212 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb882742-96cb-442a-9acd-ac72d613faa9 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:34.614499 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb882742-96cb-442a-9acd-ac72d613faa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:34.614777 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb882742-96cb-442a-9acd-ac72d613faa9 None admin] PATH_INFO: `/v3/users/864522ef4c9e4f449c3ee64cb0bf46b7` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:34.629355 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb882742-96cb-442a-9acd-ac72d613faa9 None admin] RBAC: Authorizing `identity:get_user(user_id=864522ef4c9e4f449c3ee64cb0bf46b7)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:34.630818 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb882742-96cb-442a-9acd-ac72d613faa9 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:34.632576 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 62/119] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:05:34 2022] GET /identity/v3/users/864522ef4c9e4f449c3ee64cb0bf46b7 => generated 289 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:34.632576 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73671361-a70b-45b2-8f36-6f80cdf506ea None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:34.632576 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73671361-a70b-45b2-8f36-6f80cdf506ea None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:34.632576 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73671361-a70b-45b2-8f36-6f80cdf506ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:34.652448 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 58/120] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Jul 18 06:05:34 2022] POST /identity/v3/projects => generated 302 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:05:34.652448 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47ec49-218e-4523-b91a-927279892954 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:34.654110 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-73671361-a70b-45b2-8f36-6f80cdf506ea None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:34.655475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-73671361-a70b-45b2-8f36-6f80cdf506ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:34.742960 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 63/121] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:05:34.750101 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcc092-8537-44c4-8498-4464772f339d None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:34.750401 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcc092-8537-44c4-8498-4464772f339d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:34.750692 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcc092-8537-44c4-8498-4464772f339d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:34.760901 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:34.785621 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-d2fcc092-8537-44c4-8498-4464772f339d None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:34.787225 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-d2fcc092-8537-44c4-8498-4464772f339d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:34.804454 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:34.806111 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47ec49-218e-4523-b91a-927279892954 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:34.890022 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c825963-549f-4375-b1f1-6bcf5940fcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:34.890912 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c825963-549f-4375-b1f1-6bcf5940fcbe None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:34.891205 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c825963-549f-4375-b1f1-6bcf5940fcbe None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:34.891488 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c825963-549f-4375-b1f1-6bcf5940fcbe None admin] PATH_INFO: `/v3/projects/4620d11733d2401eb2122d406221cf0d` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:34.891583 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 64/122] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:05:34.892391 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fcc092-8537-44c4-8498-4464772f339d None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:34.892926 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c825963-549f-4375-b1f1-6bcf5940fcbe None admin] RBAC: Authorizing `identity:get_project(project_id=4620d11733d2401eb2122d406221cf0d)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:34.894411 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c825963-549f-4375-b1f1-6bcf5940fcbe None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:34.896216 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 59/123] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:05:34 2022] GET /identity/v3/projects/4620d11733d2401eb2122d406221cf0d => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:34.897140 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c825963-549f-4375-b1f1-6bcf5940fcbe None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:35.010131 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fcc092-8537-44c4-8498-4464772f339d None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:35.019596 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c825963-549f-4375-b1f1-6bcf5940fcbe None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:35.059060 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9810fc62-470e-4943-8712-01439b6c5b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:35.059393 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e097097c-6c46-487e-a339-03504408c55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:35.059895 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810fc62-470e-4943-8712-01439b6c5b85 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:35.060189 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810fc62-470e-4943-8712-01439b6c5b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:35.060302 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e097097c-6c46-487e-a339-03504408c55d None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:35.060497 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810fc62-470e-4943-8712-01439b6c5b85 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:35.060637 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e097097c-6c46-487e-a339-03504408c55d None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:35.061025 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e097097c-6c46-487e-a339-03504408c55d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:35.073980 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9810fc62-470e-4943-8712-01439b6c5b85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:05:35.076788 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:35.084939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-9810fc62-470e-4943-8712-01439b6c5b85 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 06:05:35.084939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:05:35.084939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:05:35.084939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:05:35.084939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:05:35.084939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:05:35.084939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:05:35.084939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:05:35.084939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:05:35.084939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:05:35.084939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:05:35.084939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:05:35.084939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:05:35.084939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:05:35.084939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:05:35.084939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:05:35.090149 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:05:35.090149 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:05:35.090149 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:05:35.090149 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:05:35.090149 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:05:35.090149 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:05:35.090149 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:05:35.090149 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:05:35.090149 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:05:35.090149 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:05:35.090149 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:05:35.090149 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:05:35.090149 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:05:35.090149 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:05:35.090149 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:05:35.090149 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:05:35.091136 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:05:35.091136 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:05:35.091136 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:05:35.091136 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:05:35.091136 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:05:35.091136 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:05:35.091136 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:05:35.091136 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:05:35.091136 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:05:35.091136 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:05:35.091136 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 06:05:35.091136 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:05:35.091136 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 60/124] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 18 06:05:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:05:35.092885 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9810fc62-470e-4943-8712-01439b6c5b85 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:35.102271 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:35.155323 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e097097c-6c46-487e-a339-03504408c55d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:35.156854 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e097097c-6c46-487e-a339-03504408c55d None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:35.223154 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9810fc62-470e-4943-8712-01439b6c5b85 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:35.262438 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba7a7a-beac-4410-8d6e-c79a552e0f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:35.263185 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 65/125] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:05:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 06:05:35.263761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba7a7a-beac-4410-8d6e-c79a552e0f75 None admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:35.264131 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e097097c-6c46-487e-a339-03504408c55d None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:35.264342 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba7a7a-beac-4410-8d6e-c79a552e0f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:35.264793 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba7a7a-beac-4410-8d6e-c79a552e0f75 None admin] PATH_INFO: `/v3/projects/4620d11733d2401eb2122d406221cf0d/users/8783051891e642af84b75a46b4d6b15e/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:35.267327 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba7a7a-beac-4410-8d6e-c79a552e0f75 None admin] RBAC: Authorizing `identity:create_grant(project_id=4620d11733d2401eb2122d406221cf0d, user_id=8783051891e642af84b75a46b4d6b15e, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:35.269409 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba7a7a-beac-4410-8d6e-c79a552e0f75 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:35.313685 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:35.373652 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e097097c-6c46-487e-a339-03504408c55d None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:35.389533 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 61/126] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 18 06:05:35 2022] PUT /identity/v3/projects/4620d11733d2401eb2122d406221cf0d/users/8783051891e642af84b75a46b4d6b15e/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:05:35.391320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba7a7a-beac-4410-8d6e-c79a552e0f75 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:35.418949 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7034816e-3303-4108-953a-5d96404e1b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:35.420468 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7034816e-3303-4108-953a-5d96404e1b16 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:35.420897 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7034816e-3303-4108-953a-5d96404e1b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:35.421233 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7034816e-3303-4108-953a-5d96404e1b16 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:35.449182 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7034816e-3303-4108-953a-5d96404e1b16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:05:35.469630 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-7034816e-3303-4108-953a-5d96404e1b16 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 06:05:35.469630 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:05:35.469630 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:05:35.469630 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:05:35.469630 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:05:35.469630 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:05:35.469630 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:05:35.469630 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:05:35.469630 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:05:35.469630 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:05:35.469630 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:05:35.469630 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:05:35.469630 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:05:35.469630 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:05:35.469630 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:05:35.469630 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:05:35.470572 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:05:35.470572 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:05:35.470572 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:05:35.470572 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:05:35.470572 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:05:35.470572 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:05:35.470572 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:05:35.470572 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:05:35.470572 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:05:35.470572 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:05:35.470572 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:05:35.470572 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:05:35.470572 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:05:35.470572 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:05:35.470572 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:05:35.470572 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:05:35.471347 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:05:35.471347 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:05:35.471347 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:05:35.471347 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:05:35.471347 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:05:35.471347 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:05:35.471347 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:05:35.471347 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:05:35.471347 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:05:35.471347 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:05:35.471347 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 06:05:35.471347 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:05:35.472473 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 66/127] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:05:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:05:35.476054 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7034816e-3303-4108-953a-5d96404e1b16 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:35.539522 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:35.578001 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba7a7a-beac-4410-8d6e-c79a552e0f75 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:35.631454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfe57d6-e9a0-40c6-ac1a-9b7bdb12fce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:35.632399 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe57d6-e9a0-40c6-ac1a-9b7bdb12fce1 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:35.633375 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe57d6-e9a0-40c6-ac1a-9b7bdb12fce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:35.633827 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe57d6-e9a0-40c6-ac1a-9b7bdb12fce1 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:35.634755 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfe57d6-e9a0-40c6-ac1a-9b7bdb12fce1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:35.635665 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfe57d6-e9a0-40c6-ac1a-9b7bdb12fce1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:35.638118 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfe57d6-e9a0-40c6-ac1a-9b7bdb12fce1 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:35.655697 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 62/128] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:05:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:35.657533 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfe57d6-e9a0-40c6-ac1a-9b7bdb12fce1 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:35.669895 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7034816e-3303-4108-953a-5d96404e1b16 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:35.672254 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:35.711311 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb23d29f-da1a-467e-aca1-f299ea591ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:35.712236 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23d29f-da1a-467e-aca1-f299ea591ffc None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:35.712590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23d29f-da1a-467e-aca1-f299ea591ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:35.712872 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23d29f-da1a-467e-aca1-f299ea591ffc None admin] PATH_INFO: `/v3/projects/4620d11733d2401eb2122d406221cf0d` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:35.714293 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb23d29f-da1a-467e-aca1-f299ea591ffc None admin] RBAC: Authorizing `identity:get_project(project_id=4620d11733d2401eb2122d406221cf0d)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:35.716359 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb23d29f-da1a-467e-aca1-f299ea591ffc None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:35.718656 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 67/129] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:05:35 2022] GET /identity/v3/projects/4620d11733d2401eb2122d406221cf0d => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:35.719797 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb23d29f-da1a-467e-aca1-f299ea591ffc None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:35.784990 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfe57d6-e9a0-40c6-ac1a-9b7bdb12fce1 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:35.821426 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:35.828606 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c047e9-79fe-4295-82d7-e2d072b1e606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:35.830647 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c047e9-79fe-4295-82d7-e2d072b1e606 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:35.830948 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c047e9-79fe-4295-82d7-e2d072b1e606 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:35.831240 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c047e9-79fe-4295-82d7-e2d072b1e606 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:35.835069 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c047e9-79fe-4295-82d7-e2d072b1e606 None admin] RBAC: Adding query filter params (name=member) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:35.835432 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c047e9-79fe-4295-82d7-e2d072b1e606 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:35.837756 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c047e9-79fe-4295-82d7-e2d072b1e606 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:35.852976 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 63/130] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 18 06:05:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:35.856185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c047e9-79fe-4295-82d7-e2d072b1e606 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:35.864392 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb23d29f-da1a-467e-aca1-f299ea591ffc None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:35.940631 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a60c333-17d4-47f6-9301-c6549b0082f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:35.942158 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60c333-17d4-47f6-9301-c6549b0082f0 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:35.942849 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60c333-17d4-47f6-9301-c6549b0082f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:35.943471 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60c333-17d4-47f6-9301-c6549b0082f0 None admin] PATH_INFO: `/v3/users/ab532fe0550b4d79a1bc56aff3a426f0` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:35.945209 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a60c333-17d4-47f6-9301-c6549b0082f0 None admin] RBAC: Authorizing `identity:get_user(user_id=ab532fe0550b4d79a1bc56aff3a426f0)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:35.952467 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a60c333-17d4-47f6-9301-c6549b0082f0 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:35.952467 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 68/131] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:05:35 2022] GET /identity/v3/users/ab532fe0550b4d79a1bc56aff3a426f0 => generated 275 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:35.952467 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a60c333-17d4-47f6-9301-c6549b0082f0 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:36.004331 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c047e9-79fe-4295-82d7-e2d072b1e606 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:36.033488 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:36.047177 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e99d3c-d9fe-4d7e-84dc-418826b2c770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:36.049835 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e99d3c-d9fe-4d7e-84dc-418826b2c770 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:36.049835 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e99d3c-d9fe-4d7e-84dc-418826b2c770 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:36.049835 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e99d3c-d9fe-4d7e-84dc-418826b2c770 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:36.050038 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e99d3c-d9fe-4d7e-84dc-418826b2c770 None admin] RBAC: Adding query filter params (role.id=432bfb33f0224d11acc5bae2c196dc77, scope.project.id=4620d11733d2401eb2122d406221cf0d, user.id=864522ef4c9e4f449c3ee64cb0bf46b7) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:36.050490 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e99d3c-d9fe-4d7e-84dc-418826b2c770 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:36.052501 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e99d3c-d9fe-4d7e-84dc-418826b2c770 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:36.061134 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 64/132] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:05:35 2022] GET /identity/v3/role_assignments?user.id=864522ef4c9e4f449c3ee64cb0bf46b7&scope.project.id=4620d11733d2401eb2122d406221cf0d&role.id=432bfb33f0224d11acc5bae2c196dc77 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:36.063039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e99d3c-d9fe-4d7e-84dc-418826b2c770 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:36.144843 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:36.150523 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a60c333-17d4-47f6-9301-c6549b0082f0 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:36.192316 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e99d3c-d9fe-4d7e-84dc-418826b2c770 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:36.242925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee6d99-1a9b-4380-a435-7a175ca6f7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:36.243817 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee6d99-1a9b-4380-a435-7a175ca6f7fa None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:36.244102 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee6d99-1a9b-4380-a435-7a175ca6f7fa None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:36.244387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee6d99-1a9b-4380-a435-7a175ca6f7fa None admin] PATH_INFO: `/v3/users/ab532fe0550b4d79a1bc56aff3a426f0` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:36.245876 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ee6d99-1a9b-4380-a435-7a175ca6f7fa None admin] RBAC: Authorizing `identity:get_user(user_id=ab532fe0550b4d79a1bc56aff3a426f0)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:36.248315 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ee6d99-1a9b-4380-a435-7a175ca6f7fa None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:36.250138 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 69/133] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:05:35 2022] GET /identity/v3/users/ab532fe0550b4d79a1bc56aff3a426f0 => generated 275 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:36.258619 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d937045-669d-41c8-976b-8a98ca29c2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:36.259904 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee6d99-1a9b-4380-a435-7a175ca6f7fa None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:36.261261 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d937045-669d-41c8-976b-8a98ca29c2a5 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:36.262063 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d937045-669d-41c8-976b-8a98ca29c2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:36.262945 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d937045-669d-41c8-976b-8a98ca29c2a5 None admin] PATH_INFO: `/v3/projects/4620d11733d2401eb2122d406221cf0d` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:36.269370 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d937045-669d-41c8-976b-8a98ca29c2a5 None admin] RBAC: Authorizing `identity:get_project(project_id=4620d11733d2401eb2122d406221cf0d)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:36.273707 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d937045-669d-41c8-976b-8a98ca29c2a5 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:36.276852 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 65/134] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:05:36 2022] GET /identity/v3/projects/4620d11733d2401eb2122d406221cf0d => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:36.283001 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d937045-669d-41c8-976b-8a98ca29c2a5 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:36.402564 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee6d99-1a9b-4380-a435-7a175ca6f7fa None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:36.409184 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d937045-669d-41c8-976b-8a98ca29c2a5 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:36.410146 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:36.441060 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:36.450448 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2967e3bf-80c0-4231-a4ca-4679f178bec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:36.451417 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967e3bf-80c0-4231-a4ca-4679f178bec2 None admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:36.451710 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967e3bf-80c0-4231-a4ca-4679f178bec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:36.451990 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967e3bf-80c0-4231-a4ca-4679f178bec2 None admin] PATH_INFO: `/v3/projects/4620d11733d2401eb2122d406221cf0d/users/ab532fe0550b4d79a1bc56aff3a426f0/roles/04868303225144c389068779288f3a42` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:36.455142 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2967e3bf-80c0-4231-a4ca-4679f178bec2 None admin] RBAC: Authorizing `identity:create_grant(project_id=4620d11733d2401eb2122d406221cf0d, user_id=ab532fe0550b4d79a1bc56aff3a426f0, role_id=04868303225144c389068779288f3a42)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:36.460675 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2967e3bf-80c0-4231-a4ca-4679f178bec2 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:36.460789 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf0657-b9c1-4c11-a52a-6f65e64f77b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:36.460789 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf0657-b9c1-4c11-a52a-6f65e64f77b9 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:36.460789 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf0657-b9c1-4c11-a52a-6f65e64f77b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:36.460789 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf0657-b9c1-4c11-a52a-6f65e64f77b9 None admin] PATH_INFO: `/v3/projects/1a5e942e8e934867bf5c202585b4fa2b` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:36.465703 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bf0657-b9c1-4c11-a52a-6f65e64f77b9 None admin] RBAC: Authorizing `identity:get_project(project_id=1a5e942e8e934867bf5c202585b4fa2b)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:36.467412 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bf0657-b9c1-4c11-a52a-6f65e64f77b9 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:36.472475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 70/135] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:05:36 2022] GET /identity/v3/projects/1a5e942e8e934867bf5c202585b4fa2b => generated 312 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:36.478653 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf0657-b9c1-4c11-a52a-6f65e64f77b9 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:36.559037 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 66/136] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 18 06:05:36 2022] PUT /identity/v3/projects/4620d11733d2401eb2122d406221cf0d/users/ab532fe0550b4d79a1bc56aff3a426f0/roles/04868303225144c389068779288f3a42 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:05:36.628509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf0657-b9c1-4c11-a52a-6f65e64f77b9 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:36.686656 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4bc219-0e28-41c0-9bf2-76e1383e0495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:36.687496 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bc219-0e28-41c0-9bf2-76e1383e0495 None admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:36.688063 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bc219-0e28-41c0-9bf2-76e1383e0495 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:36.688063 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bc219-0e28-41c0-9bf2-76e1383e0495 None admin] PATH_INFO: `/v3/projects/1a5e942e8e934867bf5c202585b4fa2b/users/ab532fe0550b4d79a1bc56aff3a426f0/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:36.693835 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4bc219-0e28-41c0-9bf2-76e1383e0495 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a5e942e8e934867bf5c202585b4fa2b, user_id=ab532fe0550b4d79a1bc56aff3a426f0, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:36.695308 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4bc219-0e28-41c0-9bf2-76e1383e0495 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:36.768355 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 71/137] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 18 06:05:36 2022] PUT /identity/v3/projects/1a5e942e8e934867bf5c202585b4fa2b/users/ab532fe0550b4d79a1bc56aff3a426f0/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:05:36.927125 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f779eaa-68b4-471a-bf3e-cdb8a7418eeb None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:36.927654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f779eaa-68b4-471a-bf3e-cdb8a7418eeb None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:36.927938 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f779eaa-68b4-471a-bf3e-cdb8a7418eeb None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:36.929229 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 67/138] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:05:36.936151 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2d3cc-e45e-4560-aaf8-92da3690df22 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:36.936502 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2d3cc-e45e-4560-aaf8-92da3690df22 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:36.936820 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2d3cc-e45e-4560-aaf8-92da3690df22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:36.966255 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-81a2d3cc-e45e-4560-aaf8-92da3690df22 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:36.967582 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-81a2d3cc-e45e-4560-aaf8-92da3690df22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:37.008159 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:37.076445 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 72/139] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:05:37.104653 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f779eaa-68b4-471a-bf3e-cdb8a7418eeb None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:37.223806 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f779eaa-68b4-471a-bf3e-cdb8a7418eeb None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:37.262260 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79372a54-f885-4f62-aa20-0c2dd6150cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:37.263487 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79372a54-f885-4f62-aa20-0c2dd6150cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:37.264013 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79372a54-f885-4f62-aa20-0c2dd6150cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:37.264542 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79372a54-f885-4f62-aa20-0c2dd6150cb7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:37.276360 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79372a54-f885-4f62-aa20-0c2dd6150cb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:05:37.285740 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-79372a54-f885-4f62-aa20-0c2dd6150cb7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 06:05:37.285740 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:05:37.285740 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:05:37.285740 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:05:37.285740 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:05:37.285740 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:05:37.285740 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:05:37.285740 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:05:37.285740 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:05:37.285740 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:05:37.285740 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:05:37.285740 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:05:37.285740 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:05:37.285740 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:05:37.285740 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:05:37.285740 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:05:37.286736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:05:37.286736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:05:37.286736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:05:37.286736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:05:37.286736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:05:37.286736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:05:37.286736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:05:37.286736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:05:37.286736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:05:37.286736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:05:37.286736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:05:37.286736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:05:37.286736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:05:37.286736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:05:37.286736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:05:37.286736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:05:37.287519 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:05:37.287519 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:05:37.287519 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:05:37.287519 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:05:37.287519 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:05:37.287519 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:05:37.287519 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:05:37.287519 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:05:37.287519 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:05:37.287519 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:05:37.287519 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 06:05:37.287519 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:05:37.293476 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 68/140] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:05:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:05:37.297886 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79372a54-f885-4f62-aa20-0c2dd6150cb7 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:37.298780 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51540d-3f51-4872-ab10-32581073c788 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:37.299288 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51540d-3f51-4872-ab10-32581073c788 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:37.299806 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51540d-3f51-4872-ab10-32581073c788 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:37.302732 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 73/141] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:05:37.315461 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a3a2a8-df33-4b43-8ae8-a9178a9da3b0 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:37.316345 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a3a2a8-df33-4b43-8ae8-a9178a9da3b0 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:37.316876 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a3a2a8-df33-4b43-8ae8-a9178a9da3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:37.336919 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-f2a3a2a8-df33-4b43-8ae8-a9178a9da3b0 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:37.338812 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f2a3a2a8-df33-4b43-8ae8-a9178a9da3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:37.384802 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:37.422916 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 74/142] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:05:37.462025 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a3a2a8-df33-4b43-8ae8-a9178a9da3b0 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:37.546784 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79372a54-f885-4f62-aa20-0c2dd6150cb7 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:37.574999 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:37.583523 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a3a2a8-df33-4b43-8ae8-a9178a9da3b0 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:37.625378 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699fa092-19a0-4220-a5fc-b55b6ee93a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:37.626691 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699fa092-19a0-4220-a5fc-b55b6ee93a29 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:37.627402 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699fa092-19a0-4220-a5fc-b55b6ee93a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:37.627912 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699fa092-19a0-4220-a5fc-b55b6ee93a29 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:37.629417 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699fa092-19a0-4220-a5fc-b55b6ee93a29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:37.631109 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699fa092-19a0-4220-a5fc-b55b6ee93a29 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:37.633852 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 75/143] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:05:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:05:37.640932 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699fa092-19a0-4220-a5fc-b55b6ee93a29 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:37.650280 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5984e2de-c4ce-4ecd-895a-6678b3c0ffff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:37.651021 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984e2de-c4ce-4ecd-895a-6678b3c0ffff None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:37.651313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984e2de-c4ce-4ecd-895a-6678b3c0ffff None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:37.651601 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984e2de-c4ce-4ecd-895a-6678b3c0ffff None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:37.653093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5984e2de-c4ce-4ecd-895a-6678b3c0ffff None admin] RBAC: Adding query filter params (name=member) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:37.653451 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5984e2de-c4ce-4ecd-895a-6678b3c0ffff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:37.657381 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5984e2de-c4ce-4ecd-895a-6678b3c0ffff None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:37.681857 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 69/144] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 18 06:05:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:37.692328 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5984e2de-c4ce-4ecd-895a-6678b3c0ffff None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:37.836181 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5984e2de-c4ce-4ecd-895a-6678b3c0ffff None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:37.837987 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:37.884262 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db2ef4-0fe2-447e-bc4e-3d7050e3a6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:37.885187 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db2ef4-0fe2-447e-bc4e-3d7050e3a6e1 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:37.885891 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db2ef4-0fe2-447e-bc4e-3d7050e3a6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:37.886183 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db2ef4-0fe2-447e-bc4e-3d7050e3a6e1 None admin] PATH_INFO: `/v3/users/ab532fe0550b4d79a1bc56aff3a426f0` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:37.889907 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66db2ef4-0fe2-447e-bc4e-3d7050e3a6e1 None admin] RBAC: Authorizing `identity:get_user(user_id=ab532fe0550b4d79a1bc56aff3a426f0)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:37.891384 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66db2ef4-0fe2-447e-bc4e-3d7050e3a6e1 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:37.893244 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 70/145] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:05:37 2022] GET /identity/v3/users/ab532fe0550b4d79a1bc56aff3a426f0 => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:05:37.898417 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699fa092-19a0-4220-a5fc-b55b6ee93a29 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:37.898947 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db2ef4-0fe2-447e-bc4e-3d7050e3a6e1 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:38.070683 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db2ef4-0fe2-447e-bc4e-3d7050e3a6e1 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:38.089255 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f4f88-c9ed-4ba9-8630-d76091e12346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:38.090580 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f4f88-c9ed-4ba9-8630-d76091e12346 None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:38.091010 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f4f88-c9ed-4ba9-8630-d76091e12346 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:38.091434 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f4f88-c9ed-4ba9-8630-d76091e12346 None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:38.092181 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168f4f88-c9ed-4ba9-8630-d76091e12346 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:38.093710 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168f4f88-c9ed-4ba9-8630-d76091e12346 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:38.155308 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a4a037-11d5-478f-a5cc-d5863fe84c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:38.156322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a4a037-11d5-478f-a5cc-d5863fe84c84 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:38.156867 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a4a037-11d5-478f-a5cc-d5863fe84c84 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:38.157345 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a4a037-11d5-478f-a5cc-d5863fe84c84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:38.204685 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:38.227185 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 76/146] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Jul 18 06:05:37 2022] POST /identity/v3/users => generated 283 bytes in 587 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:05:38.251922 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a4a037-11d5-478f-a5cc-d5863fe84c84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:38.253513 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a4a037-11d5-478f-a5cc-d5863fe84c84 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:38.389700 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 71/147] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:05:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 06:05:38.395552 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f4f88-c9ed-4ba9-8630-d76091e12346 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:38.438239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:38.552151 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400e5b3-9b3d-41f6-a900-1ae19400cb59 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:38.552492 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400e5b3-9b3d-41f6-a900-1ae19400cb59 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:38.552757 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400e5b3-9b3d-41f6-a900-1ae19400cb59 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:38.554314 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 72/148] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:05:38.556896 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f4f88-c9ed-4ba9-8630-d76091e12346 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:38.566511 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435b458-b6dd-4a41-9203-e4dac1e73c7d None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:38.566810 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435b458-b6dd-4a41-9203-e4dac1e73c7d None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:38.567099 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435b458-b6dd-4a41-9203-e4dac1e73c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:38.595425 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-1435b458-b6dd-4a41-9203-e4dac1e73c7d None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:38.596664 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-1435b458-b6dd-4a41-9203-e4dac1e73c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:38.631824 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03c56e-2e5d-496a-a7a8-6d1714f22fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:38.632756 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03c56e-2e5d-496a-a7a8-6d1714f22fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:38.633046 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03c56e-2e5d-496a-a7a8-6d1714f22fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:38.633323 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03c56e-2e5d-496a-a7a8-6d1714f22fd8 None admin] PATH_INFO: `/v3/projects/4620d11733d2401eb2122d406221cf0d` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:38.638656 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd03c56e-2e5d-496a-a7a8-6d1714f22fd8 None admin] RBAC: Authorizing `identity:get_project(project_id=4620d11733d2401eb2122d406221cf0d)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:38.640263 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd03c56e-2e5d-496a-a7a8-6d1714f22fd8 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:38.642109 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 77/149] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:05:38 2022] GET /identity/v3/projects/4620d11733d2401eb2122d406221cf0d => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:38.648314 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03c56e-2e5d-496a-a7a8-6d1714f22fd8 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:38.712571 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:38.719074 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 73/150] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:05:38.741811 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1435b458-b6dd-4a41-9203-e4dac1e73c7d None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:38.833438 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:38.868815 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1435b458-b6dd-4a41-9203-e4dac1e73c7d None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:38.871787 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03c56e-2e5d-496a-a7a8-6d1714f22fd8 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:38.910739 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb1879-63b5-433e-83bb-ba4c5bd4689a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:38.911877 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb1879-63b5-433e-83bb-ba4c5bd4689a None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:38.912361 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb1879-63b5-433e-83bb-ba4c5bd4689a None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:38.912864 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb1879-63b5-433e-83bb-ba4c5bd4689a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:38.927102 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb1879-63b5-433e-83bb-ba4c5bd4689a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:05:38.935607 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-78cb1879-63b5-433e-83bb-ba4c5bd4689a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:05:38.935607 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:05:38.935607 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:05:38.935607 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:05:38.935607 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:05:38.935607 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:05:38.935607 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:05:38.935607 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:05:38.935607 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:05:38.935607 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:05:38.935607 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:05:38.935607 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:05:38.935607 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:05:38.935607 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:05:38.935607 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:05:38.935607 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:05:38.936513 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:05:38.936513 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:05:38.936513 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:05:38.936513 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:05:38.936513 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:05:38.936513 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:05:38.936513 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:05:38.936513 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:05:38.936513 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:05:38.936513 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:05:38.936513 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:05:38.936513 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:05:38.936513 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:05:38.936513 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:05:38.936513 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:05:38.936513 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:05:38.937289 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:05:38.937289 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:05:38.937289 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:05:38.937289 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:05:38.937289 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:05:38.937289 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:05:38.937289 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:05:38.937289 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:05:38.937289 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:05:38.937289 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:05:38.937289 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:05:38.937289 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:05:38.939675 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 74/151] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 18 06:05:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:05:38.945442 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb1879-63b5-433e-83bb-ba4c5bd4689a None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:38.981827 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59cd851-30cd-45df-8076-96a1fb41d59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:38.982504 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59cd851-30cd-45df-8076-96a1fb41d59b None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:38.982789 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59cd851-30cd-45df-8076-96a1fb41d59b None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:38.983071 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59cd851-30cd-45df-8076-96a1fb41d59b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:38.983916 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59cd851-30cd-45df-8076-96a1fb41d59b None admin] RBAC: Adding query filter params (role.id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20, scope.project.id=4620d11733d2401eb2122d406221cf0d, user.id=ab532fe0550b4d79a1bc56aff3a426f0) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:38.984231 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59cd851-30cd-45df-8076-96a1fb41d59b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:38.985700 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59cd851-30cd-45df-8076-96a1fb41d59b None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:39.007662 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 78/152] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:05:38 2022] GET /identity/v3/role_assignments?user.id=ab532fe0550b4d79a1bc56aff3a426f0&scope.project.id=4620d11733d2401eb2122d406221cf0d&role.id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 608 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:39.041823 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:39.074694 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342be8b-5e3f-4893-a635-cc88e7625a4b None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:39.075152 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342be8b-5e3f-4893-a635-cc88e7625a4b None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:39.075676 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342be8b-5e3f-4893-a635-cc88e7625a4b None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:39.082536 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 79/153] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:39 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:05:39.093526 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d86e5-3393-43fb-b565-bc4686e8ba2e None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:39.093975 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d86e5-3393-43fb-b565-bc4686e8ba2e None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:39.094423 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d86e5-3393-43fb-b565-bc4686e8ba2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:39.108737 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb1879-63b5-433e-83bb-ba4c5bd4689a None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:39.122248 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-af4d86e5-3393-43fb-b565-bc4686e8ba2e None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:39.123595 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-af4d86e5-3393-43fb-b565-bc4686e8ba2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:39.175738 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25deacd-acbe-4872-b6b0-0eec08027fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:39.176831 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25deacd-acbe-4872-b6b0-0eec08027fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:39.177359 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25deacd-acbe-4872-b6b0-0eec08027fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:39.177852 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25deacd-acbe-4872-b6b0-0eec08027fb7 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:39.179519 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25deacd-acbe-4872-b6b0-0eec08027fb7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:39.180043 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25deacd-acbe-4872-b6b0-0eec08027fb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:39.182248 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25deacd-acbe-4872-b6b0-0eec08027fb7 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:39.195573 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 75/154] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:05:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:39.201594 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25deacd-acbe-4872-b6b0-0eec08027fb7 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:39.222163 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 80/155] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:05:39.254715 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d86e5-3393-43fb-b565-bc4686e8ba2e None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:39.372657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25deacd-acbe-4872-b6b0-0eec08027fb7 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:39.408375 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:39.427765 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df32899-12f4-48f4-b428-db41f79d6578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:39.429609 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df32899-12f4-48f4-b428-db41f79d6578 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:39.430533 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df32899-12f4-48f4-b428-db41f79d6578 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:39.431020 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df32899-12f4-48f4-b428-db41f79d6578 None admin] PATH_INFO: `/v3/users/8783051891e642af84b75a46b4d6b15e` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:39.432858 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df32899-12f4-48f4-b428-db41f79d6578 None admin] RBAC: Authorizing `identity:get_user(user_id=8783051891e642af84b75a46b4d6b15e)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:39.434107 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:39.437016 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df32899-12f4-48f4-b428-db41f79d6578 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:39.440186 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 76/156] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:05:39 2022] GET /identity/v3/users/8783051891e642af84b75a46b4d6b15e => generated 247 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:39.445265 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df32899-12f4-48f4-b428-db41f79d6578 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:39.460340 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d86e5-3393-43fb-b565-bc4686e8ba2e None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:39.536158 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223055b-bc9d-4a6c-8b85-33aa8ea49054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:39.538342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223055b-bc9d-4a6c-8b85-33aa8ea49054 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:39.539085 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223055b-bc9d-4a6c-8b85-33aa8ea49054 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:39.539565 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223055b-bc9d-4a6c-8b85-33aa8ea49054 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:39.564636 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f223055b-bc9d-4a6c-8b85-33aa8ea49054 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:05:39.587317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-f223055b-bc9d-4a6c-8b85-33aa8ea49054 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 06:05:39.587317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:05:39.587317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:05:39.587317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:05:39.587317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:05:39.587317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:05:39.587317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:05:39.587317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:05:39.587317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:05:39.587317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:05:39.587317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:05:39.587317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:05:39.587317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:05:39.587317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:05:39.587317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:05:39.587317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:05:39.589027 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:05:39.589027 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:05:39.589027 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:05:39.589027 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:05:39.589027 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:05:39.589027 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:05:39.589027 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:05:39.589027 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:05:39.589027 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:05:39.589027 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:05:39.589027 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:05:39.589027 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:05:39.589027 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:05:39.589027 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:05:39.589027 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:05:39.589027 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:05:39.591717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:05:39.591717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:05:39.591717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:05:39.591717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:05:39.591717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:05:39.591717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:05:39.591717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:05:39.591717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:05:39.591717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:05:39.591717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:05:39.591717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 06:05:39.591717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:05:39.594720 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df32899-12f4-48f4-b428-db41f79d6578 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:39.597426 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 81/157] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:05:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:05:39.607165 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223055b-bc9d-4a6c-8b85-33aa8ea49054 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:39.649024 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc358e-8073-4c47-97d9-f5c771135f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:39.653527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc358e-8073-4c47-97d9-f5c771135f0a None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:39.653527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc358e-8073-4c47-97d9-f5c771135f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:39.653527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc358e-8073-4c47-97d9-f5c771135f0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:39.727276 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:39.739633 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bc358e-8073-4c47-97d9-f5c771135f0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:39.741486 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bc358e-8073-4c47-97d9-f5c771135f0a None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:39.749940 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223055b-bc9d-4a6c-8b85-33aa8ea49054 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:39.793321 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975da384-3dfa-4ee5-ad69-5938333c0423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:39.794054 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975da384-3dfa-4ee5-ad69-5938333c0423 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:39.794525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975da384-3dfa-4ee5-ad69-5938333c0423 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:39.794805 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975da384-3dfa-4ee5-ad69-5938333c0423 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:39.795862 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975da384-3dfa-4ee5-ad69-5938333c0423 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:39.796194 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975da384-3dfa-4ee5-ad69-5938333c0423 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:39.798032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975da384-3dfa-4ee5-ad69-5938333c0423 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:39.809065 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 82/158] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 18 06:05:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:05:39.817194 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975da384-3dfa-4ee5-ad69-5938333c0423 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:39.821675 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:39.891274 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 77/159] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:05:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 06:05:39.897040 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc358e-8073-4c47-97d9-f5c771135f0a None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:39.953951 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975da384-3dfa-4ee5-ad69-5938333c0423 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:39.993752 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db20ccac-1bc3-44b2-bb96-8a9023140397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:39.995265 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db20ccac-1bc3-44b2-bb96-8a9023140397 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:39.995651 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db20ccac-1bc3-44b2-bb96-8a9023140397 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:39.996654 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db20ccac-1bc3-44b2-bb96-8a9023140397 None admin] PATH_INFO: `/v3/users/864522ef4c9e4f449c3ee64cb0bf46b7` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:39.998077 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db20ccac-1bc3-44b2-bb96-8a9023140397 None admin] RBAC: Authorizing `identity:get_user(user_id=864522ef4c9e4f449c3ee64cb0bf46b7)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:39.999648 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db20ccac-1bc3-44b2-bb96-8a9023140397 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:40.002401 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 83/160] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:05:39 2022] GET /identity/v3/users/864522ef4c9e4f449c3ee64cb0bf46b7 => generated 289 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:40.009307 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db20ccac-1bc3-44b2-bb96-8a9023140397 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:40.096849 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:40.102707 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:40.114529 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc358e-8073-4c47-97d9-f5c771135f0a None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:40.143370 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db20ccac-1bc3-44b2-bb96-8a9023140397 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:40.186676 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c0720-a3fd-4a1b-8d4b-9b434127e768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:40.187754 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c0720-a3fd-4a1b-8d4b-9b434127e768 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:40.188108 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c0720-a3fd-4a1b-8d4b-9b434127e768 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:40.188500 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c0720-a3fd-4a1b-8d4b-9b434127e768 None admin] PATH_INFO: `/v3/projects/4620d11733d2401eb2122d406221cf0d` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:40.189789 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987c0720-a3fd-4a1b-8d4b-9b434127e768 None admin] RBAC: Authorizing `identity:get_project(project_id=4620d11733d2401eb2122d406221cf0d)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:40.191444 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987c0720-a3fd-4a1b-8d4b-9b434127e768 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:40.193412 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 84/161] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:05:40 2022] GET /identity/v3/projects/4620d11733d2401eb2122d406221cf0d => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:05:40.194813 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929b303c-5cd6-41cd-b834-346ebe635616 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:40.195073 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae694556-63d1-44b1-83ca-c8a4a97a07cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:40.195366 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929b303c-5cd6-41cd-b834-346ebe635616 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:40.195713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929b303c-5cd6-41cd-b834-346ebe635616 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:40.196553 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae694556-63d1-44b1-83ca-c8a4a97a07cd None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:40.197015 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae694556-63d1-44b1-83ca-c8a4a97a07cd None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:40.197129 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 85/162] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:05:40.197817 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae694556-63d1-44b1-83ca-c8a4a97a07cd None admin] PATH_INFO: `/v3/projects/4620d11733d2401eb2122d406221cf0d` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:40.199303 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae694556-63d1-44b1-83ca-c8a4a97a07cd None admin] RBAC: Authorizing `identity:get_project(project_id=4620d11733d2401eb2122d406221cf0d)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:40.201189 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae694556-63d1-44b1-83ca-c8a4a97a07cd None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:40.205988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6fa5d5-5527-403f-81a6-7c0ee4e094db None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:40.205988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6fa5d5-5527-403f-81a6-7c0ee4e094db None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:40.205988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6fa5d5-5527-403f-81a6-7c0ee4e094db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:40.208469 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 78/163] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:05:39 2022] GET /identity/v3/projects/4620d11733d2401eb2122d406221cf0d => generated 298 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:05:40.210341 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae694556-63d1-44b1-83ca-c8a4a97a07cd None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:40.228444 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-eb6fa5d5-5527-403f-81a6-7c0ee4e094db None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:40.228444 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-eb6fa5d5-5527-403f-81a6-7c0ee4e094db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:40.307107 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 86/164] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 06:05:40.308148 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6fa5d5-5527-403f-81a6-7c0ee4e094db None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:40.401580 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae694556-63d1-44b1-83ca-c8a4a97a07cd None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:40.403698 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:40.445112 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6fa5d5-5527-403f-81a6-7c0ee4e094db None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:40.482244 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce20499-ce5f-4907-b37c-b2105345b9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:40.483264 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce20499-ce5f-4907-b37c-b2105345b9fe None admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:40.483443 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce20499-ce5f-4907-b37c-b2105345b9fe None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:40.483669 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce20499-ce5f-4907-b37c-b2105345b9fe None admin] PATH_INFO: `/v3/projects/4620d11733d2401eb2122d406221cf0d/users/864522ef4c9e4f449c3ee64cb0bf46b7/roles/432bfb33f0224d11acc5bae2c196dc77` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:40.489475 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce20499-ce5f-4907-b37c-b2105345b9fe None admin] RBAC: Authorizing `identity:create_grant(project_id=4620d11733d2401eb2122d406221cf0d, user_id=864522ef4c9e4f449c3ee64cb0bf46b7, role_id=432bfb33f0224d11acc5bae2c196dc77)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:40.489475 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce20499-ce5f-4907-b37c-b2105345b9fe None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:40.498528 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270e6665-a3a6-42d6-a4ae-bf2ec79cd992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:40.499294 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e6665-a3a6-42d6-a4ae-bf2ec79cd992 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:40.499651 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e6665-a3a6-42d6-a4ae-bf2ec79cd992 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:40.499999 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e6665-a3a6-42d6-a4ae-bf2ec79cd992 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:40.500949 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270e6665-a3a6-42d6-a4ae-bf2ec79cd992 None admin] RBAC: Adding query filter params (role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc, scope.project.id=4620d11733d2401eb2122d406221cf0d, user.id=8783051891e642af84b75a46b4d6b15e) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:40.501266 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270e6665-a3a6-42d6-a4ae-bf2ec79cd992 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:40.502798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270e6665-a3a6-42d6-a4ae-bf2ec79cd992 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:40.509892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 87/165] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:05:40 2022] GET /identity/v3/role_assignments?user.id=8783051891e642af84b75a46b4d6b15e&scope.project.id=4620d11733d2401eb2122d406221cf0d&role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:40.510716 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270e6665-a3a6-42d6-a4ae-bf2ec79cd992 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:40.571157 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 79/166] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 18 06:05:40 2022] PUT /identity/v3/projects/4620d11733d2401eb2122d406221cf0d/users/864522ef4c9e4f449c3ee64cb0bf46b7/roles/432bfb33f0224d11acc5bae2c196dc77 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:05:40.574550 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1c704-9036-44c4-aa02-4f63f4cda582 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:40.575063 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1c704-9036-44c4-aa02-4f63f4cda582 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:40.580462 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1c704-9036-44c4-aa02-4f63f4cda582 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:40.582734 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 80/167] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:40 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:05:40.589122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87188114-4f5c-4629-9542-d54caf0be9b0 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:40.590295 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87188114-4f5c-4629-9542-d54caf0be9b0 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:40.590814 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87188114-4f5c-4629-9542-d54caf0be9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:40.624570 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-87188114-4f5c-4629-9542-d54caf0be9b0 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:40.626369 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-87188114-4f5c-4629-9542-d54caf0be9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:40.636529 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270e6665-a3a6-42d6-a4ae-bf2ec79cd992 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:40.653832 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:40.670811 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:40.687130 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5a591-3c56-452f-8f94-c6ff69dc9078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:40.688031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5a591-3c56-452f-8f94-c6ff69dc9078 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:40.688278 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5a591-3c56-452f-8f94-c6ff69dc9078 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:40.688580 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5a591-3c56-452f-8f94-c6ff69dc9078 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:40.707937 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b5a591-3c56-452f-8f94-c6ff69dc9078 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:05:40.725136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-e3b5a591-3c56-452f-8f94-c6ff69dc9078 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 06:05:40.725136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:05:40.725136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:05:40.725136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:05:40.725136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:05:40.725136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:05:40.725136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:05:40.725136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:05:40.725136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:05:40.725136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:05:40.725136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:05:40.725136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:05:40.725136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:05:40.725136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:05:40.725136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:05:40.725136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:05:40.726239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:05:40.726239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:05:40.726239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:05:40.726239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:05:40.726239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:05:40.726239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:05:40.726239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:05:40.726239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:05:40.726239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:05:40.726239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:05:40.726239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:05:40.726239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:05:40.726239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:05:40.726239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:05:40.726239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:05:40.726239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:05:40.731069 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:05:40.731069 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:05:40.731069 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:05:40.731069 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:05:40.731069 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:05:40.731069 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:05:40.731069 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:05:40.731069 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:05:40.731069 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:05:40.731069 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:05:40.731069 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 06:05:40.731069 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:05:40.731069 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 88/168] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 18 06:05:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:05:40.744573 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5a591-3c56-452f-8f94-c6ff69dc9078 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:40.745024 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 81/169] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:05:40.809301 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87188114-4f5c-4629-9542-d54caf0be9b0 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:40.927465 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5a591-3c56-452f-8f94-c6ff69dc9078 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:40.952105 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87188114-4f5c-4629-9542-d54caf0be9b0 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:40.969125 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f913207-2599-4ddf-a178-129625d27738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:40.969874 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f913207-2599-4ddf-a178-129625d27738 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:40.970167 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f913207-2599-4ddf-a178-129625d27738 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:40.970448 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f913207-2599-4ddf-a178-129625d27738 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:40.971217 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f913207-2599-4ddf-a178-129625d27738 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:40.971526 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f913207-2599-4ddf-a178-129625d27738 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:40.972952 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f913207-2599-4ddf-a178-129625d27738 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:40.990968 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 89/170] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:05:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:41.000372 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f913207-2599-4ddf-a178-129625d27738 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:41.002826 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964debdb-d388-43c5-a960-7ed2141447a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:41.003869 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964debdb-d388-43c5-a960-7ed2141447a2 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:41.004377 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964debdb-d388-43c5-a960-7ed2141447a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:41.004881 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964debdb-d388-43c5-a960-7ed2141447a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:41.016760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964debdb-d388-43c5-a960-7ed2141447a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:05:41.026343 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-964debdb-d388-43c5-a960-7ed2141447a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 06:05:41.026343 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:05:41.026343 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:05:41.026343 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:05:41.026343 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:05:41.026343 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:05:41.026343 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:05:41.026343 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:05:41.026343 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:05:41.026343 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:05:41.026343 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:05:41.026343 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:05:41.026343 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:05:41.026343 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:05:41.026343 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:05:41.026343 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:05:41.027803 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:05:41.027803 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:05:41.027803 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:05:41.027803 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:05:41.027803 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:05:41.027803 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:05:41.027803 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:05:41.027803 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:05:41.027803 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:05:41.027803 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:05:41.027803 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:05:41.027803 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:05:41.027803 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:05:41.027803 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:05:41.027803 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:05:41.027803 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:05:41.029315 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:05:41.029315 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:05:41.029315 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:05:41.029315 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:05:41.029315 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:05:41.029315 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:05:41.029315 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:05:41.029315 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:05:41.029315 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:05:41.029315 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:05:41.029315 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 06:05:41.029315 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:05:41.036535 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 82/171] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:05:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:05:41.038482 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964debdb-d388-43c5-a960-7ed2141447a2 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:41.057919 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:41.143339 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:41.151446 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964debdb-d388-43c5-a960-7ed2141447a2 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:41.191828 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba7cb19-a916-4e8c-9de0-c334c71df01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:41.192937 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7cb19-a916-4e8c-9de0-c334c71df01b None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:41.193428 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7cb19-a916-4e8c-9de0-c334c71df01b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:41.193910 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7cb19-a916-4e8c-9de0-c334c71df01b None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:41.194883 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba7cb19-a916-4e8c-9de0-c334c71df01b None admin] RBAC: Adding query filter params (name=member) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:41.195390 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba7cb19-a916-4e8c-9de0-c334c71df01b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:41.197138 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba7cb19-a916-4e8c-9de0-c334c71df01b None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:41.211263 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 83/172] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 18 06:05:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:41.212888 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba7cb19-a916-4e8c-9de0-c334c71df01b None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:41.274684 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f913207-2599-4ddf-a178-129625d27738 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:41.301675 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:41.320562 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba7cb19-a916-4e8c-9de0-c334c71df01b None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:41.358016 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4d479-ae87-4fee-8ff9-3d5dc40695fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:41.359054 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4d479-ae87-4fee-8ff9-3d5dc40695fd None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:41.359449 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4d479-ae87-4fee-8ff9-3d5dc40695fd None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:41.359836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4d479-ae87-4fee-8ff9-3d5dc40695fd None admin] PATH_INFO: `/v3/users/ab532fe0550b4d79a1bc56aff3a426f0` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:41.361182 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f4d479-ae87-4fee-8ff9-3d5dc40695fd None admin] RBAC: Authorizing `identity:get_user(user_id=ab532fe0550b4d79a1bc56aff3a426f0)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:41.362744 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f4d479-ae87-4fee-8ff9-3d5dc40695fd None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:41.364562 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 84/173] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:05:41 2022] GET /identity/v3/users/ab532fe0550b4d79a1bc56aff3a426f0 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:05:41.370255 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4d479-ae87-4fee-8ff9-3d5dc40695fd None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:41.381995 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d05c9f-dd50-4a9e-a289-d1a2415cf1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:41.382898 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d05c9f-dd50-4a9e-a289-d1a2415cf1d0 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:41.383182 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d05c9f-dd50-4a9e-a289-d1a2415cf1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:41.383459 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d05c9f-dd50-4a9e-a289-d1a2415cf1d0 None admin] PATH_INFO: `/v3/users/ab532fe0550b4d79a1bc56aff3a426f0` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:41.389182 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d05c9f-dd50-4a9e-a289-d1a2415cf1d0 None admin] RBAC: Authorizing `identity:get_user(user_id=ab532fe0550b4d79a1bc56aff3a426f0)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:41.390653 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d05c9f-dd50-4a9e-a289-d1a2415cf1d0 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:41.392676 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 90/174] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:05:40 2022] GET /identity/v3/users/ab532fe0550b4d79a1bc56aff3a426f0 => generated 275 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:41.409405 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d05c9f-dd50-4a9e-a289-d1a2415cf1d0 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:41.481241 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4d479-ae87-4fee-8ff9-3d5dc40695fd None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:41.517419 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588b039-ce97-4c16-ad22-e686a55009d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:41.518199 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588b039-ce97-4c16-ad22-e686a55009d1 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:41.518542 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588b039-ce97-4c16-ad22-e686a55009d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:41.518899 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588b039-ce97-4c16-ad22-e686a55009d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:41.531944 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:41.594110 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4588b039-ce97-4c16-ad22-e686a55009d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:41.595753 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4588b039-ce97-4c16-ad22-e686a55009d1 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:41.711900 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 85/175] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:05:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 06:05:41.713492 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588b039-ce97-4c16-ad22-e686a55009d1 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:41.722639 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d05c9f-dd50-4a9e-a289-d1a2415cf1d0 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:41.761873 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:41.794837 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:41.820610 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588b039-ce97-4c16-ad22-e686a55009d1 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:41.829233 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3bc598-db04-4086-b913-141a85f13742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:41.830276 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bc598-db04-4086-b913-141a85f13742 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:41.830764 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bc598-db04-4086-b913-141a85f13742 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:41.831239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bc598-db04-4086-b913-141a85f13742 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:41.858842 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28832a-355e-4700-9486-e3d2147d3809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:41.859722 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28832a-355e-4700-9486-e3d2147d3809 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:41.860009 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28832a-355e-4700-9486-e3d2147d3809 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:41.860288 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28832a-355e-4700-9486-e3d2147d3809 None admin] PATH_INFO: `/v3/projects/1a5e942e8e934867bf5c202585b4fa2b` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:41.861527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f28832a-355e-4700-9486-e3d2147d3809 None admin] RBAC: Authorizing `identity:get_project(project_id=1a5e942e8e934867bf5c202585b4fa2b)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:41.863083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f28832a-355e-4700-9486-e3d2147d3809 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:41.869959 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 86/176] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:05:41 2022] GET /identity/v3/projects/1a5e942e8e934867bf5c202585b4fa2b => generated 312 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:05:41.871789 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28832a-355e-4700-9486-e3d2147d3809 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:41.963520 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3bc598-db04-4086-b913-141a85f13742 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:41.965962 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3bc598-db04-4086-b913-141a85f13742 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:41.980243 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28832a-355e-4700-9486-e3d2147d3809 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:41.988992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:42.016957 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24dbffc-7f09-4efa-ba4f-363b9003a28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:42.017633 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24dbffc-7f09-4efa-ba4f-363b9003a28c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:42.017920 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24dbffc-7f09-4efa-ba4f-363b9003a28c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:42.018499 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24dbffc-7f09-4efa-ba4f-363b9003a28c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:42.019047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24dbffc-7f09-4efa-ba4f-363b9003a28c None admin] RBAC: Adding query filter params (role.id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20, scope.project.id=1a5e942e8e934867bf5c202585b4fa2b, user.id=ab532fe0550b4d79a1bc56aff3a426f0) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:42.019363 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24dbffc-7f09-4efa-ba4f-363b9003a28c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:42.020854 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24dbffc-7f09-4efa-ba4f-363b9003a28c None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:42.032547 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 87/177] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:05:41 2022] GET /identity/v3/role_assignments?user.id=ab532fe0550b4d79a1bc56aff3a426f0&scope.project.id=1a5e942e8e934867bf5c202585b4fa2b&role.id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:42.086686 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 91/178] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:05:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 678 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 06:05:42.093225 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24dbffc-7f09-4efa-ba4f-363b9003a28c None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:42.253711 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24dbffc-7f09-4efa-ba4f-363b9003a28c None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:42.264518 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0dd9c6-6c10-4d6f-895e-ec6b74a0a439 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:42.265213 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0dd9c6-6c10-4d6f-895e-ec6b74a0a439 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:42.265874 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0dd9c6-6c10-4d6f-895e-ec6b74a0a439 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:42.267697 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 92/179] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:05:42.281238 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1098f1-1474-45cd-9552-06960bb13d33 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:42.281875 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1098f1-1474-45cd-9552-06960bb13d33 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:42.282453 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1098f1-1474-45cd-9552-06960bb13d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:42.315338 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-2c1098f1-1474-45cd-9552-06960bb13d33 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:42.315338 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-2c1098f1-1474-45cd-9552-06960bb13d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:42.329075 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09769a68-f822-414c-abbb-dbc480ed4196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:42.329663 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09769a68-f822-414c-abbb-dbc480ed4196 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:42.329956 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09769a68-f822-414c-abbb-dbc480ed4196 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:42.330238 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09769a68-f822-414c-abbb-dbc480ed4196 None admin] PATH_INFO: `/v3/projects/4620d11733d2401eb2122d406221cf0d` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:42.331461 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09769a68-f822-414c-abbb-dbc480ed4196 None admin] RBAC: Authorizing `identity:get_project(project_id=4620d11733d2401eb2122d406221cf0d)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:42.337934 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:42.341147 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09769a68-f822-414c-abbb-dbc480ed4196 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:42.348440 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 88/180] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:05:42 2022] GET /identity/v3/projects/4620d11733d2401eb2122d406221cf0d => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:42.351056 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09769a68-f822-414c-abbb-dbc480ed4196 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:42.421784 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:42.464790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 93/181] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:05:42.496209 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1098f1-1474-45cd-9552-06960bb13d33 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:42.537001 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09769a68-f822-414c-abbb-dbc480ed4196 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:42.601203 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d97f32c-c28f-497e-828e-bc40e59234ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:42.602760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97f32c-c28f-497e-828e-bc40e59234ae None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:42.603330 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97f32c-c28f-497e-828e-bc40e59234ae None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:42.604060 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97f32c-c28f-497e-828e-bc40e59234ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:42.607376 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d97f32c-c28f-497e-828e-bc40e59234ae None admin] RBAC: Adding query filter params (role.id=04868303225144c389068779288f3a42, scope.project.id=4620d11733d2401eb2122d406221cf0d, user.id=ab532fe0550b4d79a1bc56aff3a426f0) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:42.608168 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d97f32c-c28f-497e-828e-bc40e59234ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:42.611034 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d97f32c-c28f-497e-828e-bc40e59234ae None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:42.623221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 89/182] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:05:42 2022] GET /identity/v3/role_assignments?user.id=ab532fe0550b4d79a1bc56aff3a426f0&scope.project.id=4620d11733d2401eb2122d406221cf0d&role.id=04868303225144c389068779288f3a42 => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:05:42.687817 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1098f1-1474-45cd-9552-06960bb13d33 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:42.729756 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be47828-1d14-4f13-95da-2910e36bbe97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:42.730880 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be47828-1d14-4f13-95da-2910e36bbe97 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:42.731579 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be47828-1d14-4f13-95da-2910e36bbe97 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:42.731993 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be47828-1d14-4f13-95da-2910e36bbe97 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:42.746775 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be47828-1d14-4f13-95da-2910e36bbe97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:05:43.063825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-1be47828-1d14-4f13-95da-2910e36bbe97 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:05:43.063825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:05:43.063825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:05:43.063825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:05:43.063825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:05:43.063825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:05:43.063825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:05:43.063825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:05:43.063825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:05:43.063825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:05:43.063825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:05:43.063825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:05:43.063825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:05:43.063825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:05:43.063825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:05:43.063825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:05:43.065908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:05:43.065908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:05:43.065908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:05:43.065908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:05:43.065908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:05:43.065908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:05:43.065908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:05:43.065908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:05:43.065908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:05:43.065908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:05:43.065908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:05:43.065908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:05:43.065908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:05:43.065908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:05:43.065908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:05:43.065908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:05:43.067708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:05:43.067708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:05:43.067708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:05:43.067708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:05:43.067708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:05:43.067708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:05:43.067708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:05:43.067708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:05:43.067708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:05:43.067708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:05:43.067708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:05:43.067708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:05:43.067708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 94/183] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 18 06:05:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:05:43.069419 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d97f32c-c28f-497e-828e-bc40e59234ae None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:43.069419 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d97f32c-c28f-497e-828e-bc40e59234ae None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:43.069419 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:43.069858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6eb774-e5a9-4c8c-ac66-2523cc0808d4 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:43.069858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6eb774-e5a9-4c8c-ac66-2523cc0808d4 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:43.069858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6eb774-e5a9-4c8c-ac66-2523cc0808d4 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:43.069858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 95/184] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:05:43.069858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde20496-8df8-4ae1-9aa9-d2edeac9b9fe None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:43.069858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde20496-8df8-4ae1-9aa9-d2edeac9b9fe None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:43.070779 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde20496-8df8-4ae1-9aa9-d2edeac9b9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:43.070930 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc23986-6560-4e05-864e-8aeb55020293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:43.070930 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc23986-6560-4e05-864e-8aeb55020293 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:43.070930 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc23986-6560-4e05-864e-8aeb55020293 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:43.070930 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc23986-6560-4e05-864e-8aeb55020293 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:43.071485 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc23986-6560-4e05-864e-8aeb55020293 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:43.071485 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc23986-6560-4e05-864e-8aeb55020293 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:43.071485 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc23986-6560-4e05-864e-8aeb55020293 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:43.071936 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-fde20496-8df8-4ae1-9aa9-d2edeac9b9fe None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:43.071936 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-fde20496-8df8-4ae1-9aa9-d2edeac9b9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:43.071936 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:43.072347 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 90/185] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:05:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:05:43.072347 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc23986-6560-4e05-864e-8aeb55020293 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:43.072542 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 96/186] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:05:43.077387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde20496-8df8-4ae1-9aa9-d2edeac9b9fe None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:43.149507 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc23986-6560-4e05-864e-8aeb55020293 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:43.212071 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d91284c-7d19-4296-8243-f2d007998627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:43.213995 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91284c-7d19-4296-8243-f2d007998627 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:43.215227 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91284c-7d19-4296-8243-f2d007998627 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:43.215953 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91284c-7d19-4296-8243-f2d007998627 None admin] PATH_INFO: `/v3/users/c44e70d11cad4202a68a87daf04ec41b` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:43.216966 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde20496-8df8-4ae1-9aa9-d2edeac9b9fe None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:43.242611 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d91284c-7d19-4296-8243-f2d007998627 None admin] RBAC: Authorizing `identity:get_user(user_id=c44e70d11cad4202a68a87daf04ec41b)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:43.245452 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d91284c-7d19-4296-8243-f2d007998627 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:43.249350 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 91/187] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:05:42 2022] GET /identity/v3/users/c44e70d11cad4202a68a87daf04ec41b => generated 283 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:43.251685 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad0c887-ac66-4140-8ca2-c43549cf2bcc None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:43.252195 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad0c887-ac66-4140-8ca2-c43549cf2bcc None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:43.252701 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad0c887-ac66-4140-8ca2-c43549cf2bcc None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:43.254994 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 92/188] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:05:43.256365 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad0c887-ac66-4140-8ca2-c43549cf2bcc None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:43.271178 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc01d8c-850e-4b65-828c-ff0057edd0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:43.272653 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc01d8c-850e-4b65-828c-ff0057edd0d9 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:43.273345 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc01d8c-850e-4b65-828c-ff0057edd0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:43.273910 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc01d8c-850e-4b65-828c-ff0057edd0d9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:43.287375 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:43.300156 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc01d8c-850e-4b65-828c-ff0057edd0d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:05:43.309136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-0fc01d8c-850e-4b65-828c-ff0057edd0d9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 06:05:43.309136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:05:43.309136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:05:43.309136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:05:43.309136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:05:43.309136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:05:43.309136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:05:43.309136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:05:43.309136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:05:43.309136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:05:43.309136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:05:43.309136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:05:43.309136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:05:43.309136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:05:43.309136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:05:43.309136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:05:43.316713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:05:43.316713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:05:43.316713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:05:43.316713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:05:43.316713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:05:43.316713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:05:43.316713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:05:43.316713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:05:43.316713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:05:43.316713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:05:43.316713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:05:43.316713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:05:43.316713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:05:43.316713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:05:43.316713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:05:43.316713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:05:43.318239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:05:43.318239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:05:43.318239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:05:43.318239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:05:43.318239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:05:43.318239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:05:43.318239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:05:43.318239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:05:43.318239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:05:43.318239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:05:43.318239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 06:05:43.318239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:05:43.318239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 97/189] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 18 06:05:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:05:43.318239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf471c-c8df-478e-9576-5a9eda4a1a48 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:43.319095 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf471c-c8df-478e-9576-5a9eda4a1a48 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:43.319095 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf471c-c8df-478e-9576-5a9eda4a1a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:43.319095 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:43.331961 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-addf471c-c8df-478e-9576-5a9eda4a1a48 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:43.333517 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-addf471c-c8df-478e-9576-5a9eda4a1a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:43.390495 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad0c887-ac66-4140-8ca2-c43549cf2bcc None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:43.409857 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 98/190] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:05:43.411115 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf471c-c8df-478e-9576-5a9eda4a1a48 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:43.439387 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38623e9-1e9e-4415-9be8-c96b85ef6635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:43.440057 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38623e9-1e9e-4415-9be8-c96b85ef6635 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:43.440355 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38623e9-1e9e-4415-9be8-c96b85ef6635 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:43.440668 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38623e9-1e9e-4415-9be8-c96b85ef6635 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:43.518738 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38623e9-1e9e-4415-9be8-c96b85ef6635 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:43.520257 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38623e9-1e9e-4415-9be8-c96b85ef6635 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:43.547781 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:43.548577 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf471c-c8df-478e-9576-5a9eda4a1a48 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:43.584289 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:43.589651 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d34eed4-5d2c-436b-9e3e-589cba546bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:43.590642 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d34eed4-5d2c-436b-9e3e-589cba546bde None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:43.590923 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d34eed4-5d2c-436b-9e3e-589cba546bde None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:43.591208 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d34eed4-5d2c-436b-9e3e-589cba546bde None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:43.591995 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d34eed4-5d2c-436b-9e3e-589cba546bde None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:43.592369 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d34eed4-5d2c-436b-9e3e-589cba546bde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:43.593797 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d34eed4-5d2c-436b-9e3e-589cba546bde None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:43.603290 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 99/191] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:05:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:43.603966 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d34eed4-5d2c-436b-9e3e-589cba546bde None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:43.651527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 93/192] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:05:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 06:05:43.651527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38623e9-1e9e-4415-9be8-c96b85ef6635 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:43.732631 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d34eed4-5d2c-436b-9e3e-589cba546bde None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:43.761940 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38623e9-1e9e-4415-9be8-c96b85ef6635 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:43.767089 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b657c4b-a337-4d51-9d76-973de2f4ecf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:43.767888 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b657c4b-a337-4d51-9d76-973de2f4ecf7 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:43.768176 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b657c4b-a337-4d51-9d76-973de2f4ecf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:43.768568 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b657c4b-a337-4d51-9d76-973de2f4ecf7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:43.775384 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b657c4b-a337-4d51-9d76-973de2f4ecf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:43.775384 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b657c4b-a337-4d51-9d76-973de2f4ecf7 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:43.775384 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 100/193] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:05:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:43.775384 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b657c4b-a337-4d51-9d76-973de2f4ecf7 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:43.787515 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:43.799708 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5b95f3-b9a0-4dde-b67b-a235d2317fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:43.800622 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b95f3-b9a0-4dde-b67b-a235d2317fad None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:43.800905 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b95f3-b9a0-4dde-b67b-a235d2317fad None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:43.801181 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b95f3-b9a0-4dde-b67b-a235d2317fad None admin] PATH_INFO: `/v3/users/c44e70d11cad4202a68a87daf04ec41b` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:43.802416 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5b95f3-b9a0-4dde-b67b-a235d2317fad None admin] RBAC: Authorizing `identity:get_user(user_id=c44e70d11cad4202a68a87daf04ec41b)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:43.803883 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5b95f3-b9a0-4dde-b67b-a235d2317fad None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:43.805547 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 94/194] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:05:43 2022] GET /identity/v3/users/c44e70d11cad4202a68a87daf04ec41b => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:43.806911 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2e92d-fc3b-4df0-a14f-6cab710a0ba1 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:43.807201 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2e92d-fc3b-4df0-a14f-6cab710a0ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:43.807480 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2e92d-fc3b-4df0-a14f-6cab710a0ba1 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:43.810522 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 95/195] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:05:43.810522 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee2e92d-fc3b-4df0-a14f-6cab710a0ba1 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:43.842190 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:43.880834 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b657c4b-a337-4d51-9d76-973de2f4ecf7 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:43.914307 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee2e92d-fc3b-4df0-a14f-6cab710a0ba1 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:43.914816 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2551b8b9-bd47-4976-b455-693c2e015295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:43.915691 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551b8b9-bd47-4976-b455-693c2e015295 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:43.915979 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551b8b9-bd47-4976-b455-693c2e015295 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:43.916257 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551b8b9-bd47-4976-b455-693c2e015295 None admin] PATH_INFO: `/v3/projects/d34db96dd82a425d90973717021e4c95` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:43.917515 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2551b8b9-bd47-4976-b455-693c2e015295 None admin] RBAC: Authorizing `identity:get_project(project_id=d34db96dd82a425d90973717021e4c95)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:43.918996 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2551b8b9-bd47-4976-b455-693c2e015295 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:43.920918 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 101/196] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:05:43 2022] GET /identity/v3/projects/d34db96dd82a425d90973717021e4c95 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:43.921525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2551b8b9-bd47-4976-b455-693c2e015295 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:43.954469 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a10c6-2dde-4151-861d-09279fabd1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:43.955161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a10c6-2dde-4151-861d-09279fabd1c4 None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:43.955452 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a10c6-2dde-4151-861d-09279fabd1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:43.955744 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a10c6-2dde-4151-861d-09279fabd1c4 None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:43.956490 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851a10c6-2dde-4151-861d-09279fabd1c4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:43.957965 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851a10c6-2dde-4151-861d-09279fabd1c4 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:44.021848 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2551b8b9-bd47-4976-b455-693c2e015295 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:44.049869 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:44.055530 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74214fb-5f27-4a30-af46-2a293a0d32f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:44.056233 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74214fb-5f27-4a30-af46-2a293a0d32f8 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:44.056570 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74214fb-5f27-4a30-af46-2a293a0d32f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:44.056907 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74214fb-5f27-4a30-af46-2a293a0d32f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:44.062396 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 96/197] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Jul 18 06:05:43 2022] POST /identity/v3/users => generated 297 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:05:44.063867 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110e1c1-2983-408e-b9d8-a4c2af3c989a None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:44.064155 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110e1c1-2983-408e-b9d8-a4c2af3c989a None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:44.064500 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110e1c1-2983-408e-b9d8-a4c2af3c989a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:44.085863 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-a110e1c1-2983-408e-b9d8-a4c2af3c989a None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:44.087197 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-a110e1c1-2983-408e-b9d8-a4c2af3c989a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:44.134894 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:44.147919 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74214fb-5f27-4a30-af46-2a293a0d32f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:44.150541 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74214fb-5f27-4a30-af46-2a293a0d32f8 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:44.162271 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 97/198] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:05:44.163593 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a110e1c1-2983-408e-b9d8-a4c2af3c989a None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:44.288816 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a110e1c1-2983-408e-b9d8-a4c2af3c989a None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:44.298903 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 102/199] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:05:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 06:05:44.300745 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74214fb-5f27-4a30-af46-2a293a0d32f8 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:44.336827 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852dbdbb-a809-4f83-9107-aac8cbf0a61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:44.337514 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852dbdbb-a809-4f83-9107-aac8cbf0a61b None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:44.337799 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852dbdbb-a809-4f83-9107-aac8cbf0a61b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:44.338078 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852dbdbb-a809-4f83-9107-aac8cbf0a61b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:44.338930 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852dbdbb-a809-4f83-9107-aac8cbf0a61b None admin] RBAC: Adding query filter params (role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc, scope.project.id=d34db96dd82a425d90973717021e4c95, user.id=c44e70d11cad4202a68a87daf04ec41b) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:44.339255 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852dbdbb-a809-4f83-9107-aac8cbf0a61b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:44.340853 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852dbdbb-a809-4f83-9107-aac8cbf0a61b None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:44.348752 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 98/200] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:05:44 2022] GET /identity/v3/role_assignments?user.id=c44e70d11cad4202a68a87daf04ec41b&scope.project.id=d34db96dd82a425d90973717021e4c95&role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:44.349334 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:44.349621 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852dbdbb-a809-4f83-9107-aac8cbf0a61b None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:44.404534 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:44.436909 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74214fb-5f27-4a30-af46-2a293a0d32f8 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:44.472899 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852dbdbb-a809-4f83-9107-aac8cbf0a61b None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:44.479443 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3247a6-5b59-4a51-b658-51d60970f50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:44.481122 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3247a6-5b59-4a51-b658-51d60970f50d None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:44.481555 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3247a6-5b59-4a51-b658-51d60970f50d None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:44.482137 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3247a6-5b59-4a51-b658-51d60970f50d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:44.495126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3247a6-5b59-4a51-b658-51d60970f50d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:05:44.505537 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-0b3247a6-5b59-4a51-b658-51d60970f50d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 06:05:44.505537 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:05:44.505537 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:05:44.505537 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:05:44.505537 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:05:44.505537 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:05:44.505537 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:05:44.505537 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:05:44.505537 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:05:44.505537 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:05:44.505537 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:05:44.505537 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:05:44.505537 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:05:44.505537 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:05:44.505537 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:05:44.505537 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:05:44.506505 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:05:44.506505 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:05:44.506505 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:05:44.506505 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:05:44.506505 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:05:44.506505 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:05:44.506505 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:05:44.506505 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:05:44.506505 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:05:44.506505 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:05:44.506505 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:05:44.506505 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:05:44.506505 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:05:44.506505 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:05:44.506505 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:05:44.506505 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:05:44.507286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:05:44.507286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:05:44.507286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:05:44.507286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:05:44.507286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:05:44.507286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:05:44.507286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:05:44.507286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:05:44.507286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:05:44.507286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:05:44.507286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 06:05:44.507286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:05:44.507286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 103/201] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:05:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:05:44.512626 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3247a6-5b59-4a51-b658-51d60970f50d None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:44.522192 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c60af2-d0b7-43b3-b873-3453a1b92d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:44.523397 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c60af2-d0b7-43b3-b873-3453a1b92d0e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:44.523884 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c60af2-d0b7-43b3-b873-3453a1b92d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:44.524351 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c60af2-d0b7-43b3-b873-3453a1b92d0e None admin] PATH_INFO: `/v3/projects/d34db96dd82a425d90973717021e4c95` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:44.526075 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c60af2-d0b7-43b3-b873-3453a1b92d0e None admin] RBAC: Authorizing `identity:get_project(project_id=d34db96dd82a425d90973717021e4c95)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:44.528107 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c60af2-d0b7-43b3-b873-3453a1b92d0e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:44.530413 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 99/202] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:05:44 2022] GET /identity/v3/projects/d34db96dd82a425d90973717021e4c95 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:44.536345 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c60af2-d0b7-43b3-b873-3453a1b92d0e None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:44.646474 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:44.659694 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3247a6-5b59-4a51-b658-51d60970f50d None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:44.726177 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba5e53-1d52-46fe-b049-9336a8dd254e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:44.726983 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba5e53-1d52-46fe-b049-9336a8dd254e None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:44.727404 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba5e53-1d52-46fe-b049-9336a8dd254e None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:44.727676 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba5e53-1d52-46fe-b049-9336a8dd254e None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:44.728549 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba5e53-1d52-46fe-b049-9336a8dd254e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:44.729000 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba5e53-1d52-46fe-b049-9336a8dd254e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:44.731241 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba5e53-1d52-46fe-b049-9336a8dd254e None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:44.737341 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c60af2-d0b7-43b3-b873-3453a1b92d0e None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:44.749094 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:44.749509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 104/203] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 18 06:05:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:05:44.757499 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba5e53-1d52-46fe-b049-9336a8dd254e None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:44.782411 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5aed6d-0be7-46ba-880e-670b5f92cdf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:44.783082 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5aed6d-0be7-46ba-880e-670b5f92cdf8 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:44.783364 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5aed6d-0be7-46ba-880e-670b5f92cdf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:44.783638 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5aed6d-0be7-46ba-880e-670b5f92cdf8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:44.784505 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5aed6d-0be7-46ba-880e-670b5f92cdf8 None admin] RBAC: Adding query filter params (role.id=04868303225144c389068779288f3a42, scope.project.id=d34db96dd82a425d90973717021e4c95, user.id=c44e70d11cad4202a68a87daf04ec41b) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:44.784822 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5aed6d-0be7-46ba-880e-670b5f92cdf8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:44.786280 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5aed6d-0be7-46ba-880e-670b5f92cdf8 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:44.800042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 100/204] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:05:44 2022] GET /identity/v3/role_assignments?user.id=c44e70d11cad4202a68a87daf04ec41b&scope.project.id=d34db96dd82a425d90973717021e4c95&role.id=04868303225144c389068779288f3a42 => generated 259 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:05:44.947352 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba5e53-1d52-46fe-b049-9336a8dd254e None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:44.992180 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c12494e-8a50-4922-bb2a-e736cf5db100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:44.993190 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c12494e-8a50-4922-bb2a-e736cf5db100 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:44.993475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c12494e-8a50-4922-bb2a-e736cf5db100 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:44.993859 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c12494e-8a50-4922-bb2a-e736cf5db100 None admin] PATH_INFO: `/v3/users/864522ef4c9e4f449c3ee64cb0bf46b7` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:45.001595 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c12494e-8a50-4922-bb2a-e736cf5db100 None admin] RBAC: Authorizing `identity:get_user(user_id=864522ef4c9e4f449c3ee64cb0bf46b7)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:45.003100 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c12494e-8a50-4922-bb2a-e736cf5db100 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:45.005432 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 105/205] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:05:44 2022] GET /identity/v3/users/864522ef4c9e4f449c3ee64cb0bf46b7 => generated 289 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:45.010965 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5aed6d-0be7-46ba-880e-670b5f92cdf8 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:45.117287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5aed6d-0be7-46ba-880e-670b5f92cdf8 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:45.153638 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed4d6a0-38c1-42f3-9c04-62e05732c262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:45.154301 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4d6a0-38c1-42f3-9c04-62e05732c262 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:45.154587 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4d6a0-38c1-42f3-9c04-62e05732c262 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:45.154866 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4d6a0-38c1-42f3-9c04-62e05732c262 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:45.204877 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:45.226810 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed4d6a0-38c1-42f3-9c04-62e05732c262 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:45.228349 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed4d6a0-38c1-42f3-9c04-62e05732c262 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:45.336786 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 101/206] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:05:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 06:05:45.349788 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c12494e-8a50-4922-bb2a-e736cf5db100 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:45.379331 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:45.463270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c12494e-8a50-4922-bb2a-e736cf5db100 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:45.503889 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c53b23-c51c-4930-98bb-c00012e16fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:45.504793 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c53b23-c51c-4930-98bb-c00012e16fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:45.505078 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c53b23-c51c-4930-98bb-c00012e16fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:45.505354 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c53b23-c51c-4930-98bb-c00012e16fe5 None admin] PATH_INFO: `/v3/projects/4620d11733d2401eb2122d406221cf0d` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:45.509107 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c53b23-c51c-4930-98bb-c00012e16fe5 None admin] RBAC: Authorizing `identity:get_project(project_id=4620d11733d2401eb2122d406221cf0d)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:45.510552 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c53b23-c51c-4930-98bb-c00012e16fe5 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:45.512298 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 106/207] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:05:45 2022] GET /identity/v3/projects/4620d11733d2401eb2122d406221cf0d => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:45.518362 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed4d6a0-38c1-42f3-9c04-62e05732c262 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:45.608134 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:45.640706 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed4d6a0-38c1-42f3-9c04-62e05732c262 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:45.677042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01fbff6-7f1f-4279-be10-2f5f98a0f4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:45.677839 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01fbff6-7f1f-4279-be10-2f5f98a0f4d0 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:45.678123 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01fbff6-7f1f-4279-be10-2f5f98a0f4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:45.678402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01fbff6-7f1f-4279-be10-2f5f98a0f4d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:45.679310 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01fbff6-7f1f-4279-be10-2f5f98a0f4d0 None admin] RBAC: Adding query filter params (role.id=432bfb33f0224d11acc5bae2c196dc77, scope.project.id=4620d11733d2401eb2122d406221cf0d, user.id=864522ef4c9e4f449c3ee64cb0bf46b7) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:45.679624 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01fbff6-7f1f-4279-be10-2f5f98a0f4d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:45.681175 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01fbff6-7f1f-4279-be10-2f5f98a0f4d0 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:45.689701 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 102/208] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:05:45 2022] GET /identity/v3/role_assignments?user.id=864522ef4c9e4f449c3ee64cb0bf46b7&scope.project.id=4620d11733d2401eb2122d406221cf0d&role.id=432bfb33f0224d11acc5bae2c196dc77 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:46.749171 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c703d-6fcc-4315-bce3-b6aeff7ce7a7 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:46.749682 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c703d-6fcc-4315-bce3-b6aeff7ce7a7 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:46.749783 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c703d-6fcc-4315-bce3-b6aeff7ce7a7 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:46.751350 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 107/209] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:05:46.752309 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0061cb60-952e-4a54-af23-61bd1149ba3e None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:46.752752 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0061cb60-952e-4a54-af23-61bd1149ba3e None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:46.752918 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0061cb60-952e-4a54-af23-61bd1149ba3e None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:46.754630 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 103/210] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:05:46.757914 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b7755-f3e2-488c-9e74-0dfddffc7845 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:46.758334 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b7755-f3e2-488c-9e74-0dfddffc7845 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:46.758619 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b7755-f3e2-488c-9e74-0dfddffc7845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:46.759752 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222480f2-9477-4f8a-a299-2bd5c2d9fbd9 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:46.760171 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222480f2-9477-4f8a-a299-2bd5c2d9fbd9 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:46.760501 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222480f2-9477-4f8a-a299-2bd5c2d9fbd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:46.778694 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-222480f2-9477-4f8a-a299-2bd5c2d9fbd9 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:46.779473 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-1c8b7755-f3e2-488c-9e74-0dfddffc7845 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:46.779909 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-222480f2-9477-4f8a-a299-2bd5c2d9fbd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:46.781135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-1c8b7755-f3e2-488c-9e74-0dfddffc7845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:46.852379 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 104/211] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:05:46.860634 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c6ef9-be30-45f0-9777-235775993528 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:46.860634 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c6ef9-be30-45f0-9777-235775993528 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:46.860634 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c6ef9-be30-45f0-9777-235775993528 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:46.860634 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 105/212] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:05:46.862007 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3eaa6b-0648-4f3f-b1e3-eb1545683261 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:46.862304 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3eaa6b-0648-4f3f-b1e3-eb1545683261 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:46.862575 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3eaa6b-0648-4f3f-b1e3-eb1545683261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:46.867154 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 108/213] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:05:46.877645 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b7755-f3e2-488c-9e74-0dfddffc7845 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:46.883527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-dd3eaa6b-0648-4f3f-b1e3-eb1545683261 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:46.885010 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-dd3eaa6b-0648-4f3f-b1e3-eb1545683261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:46.897982 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:46.959313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:46.976496 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 106/214] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:05:46.976496 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3eaa6b-0648-4f3f-b1e3-eb1545683261 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:46.999291 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b7755-f3e2-488c-9e74-0dfddffc7845 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:47.038536 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358f690-93a2-4edc-b177-6a1723693cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:47.039337 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358f690-93a2-4edc-b177-6a1723693cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:47.039623 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358f690-93a2-4edc-b177-6a1723693cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:47.039901 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358f690-93a2-4edc-b177-6a1723693cf8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:47.051099 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3358f690-93a2-4edc-b177-6a1723693cf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:05:47.058871 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-3358f690-93a2-4edc-b177-6a1723693cf8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 06:05:47.058871 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:05:47.058871 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:05:47.058871 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:05:47.058871 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:05:47.058871 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:05:47.058871 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:05:47.058871 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:05:47.058871 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:05:47.058871 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:05:47.058871 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:05:47.058871 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:05:47.058871 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:05:47.058871 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:05:47.058871 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:05:47.058871 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:05:47.059769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:05:47.059769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:05:47.059769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:05:47.059769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:05:47.059769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:05:47.059769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:05:47.059769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:05:47.059769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:05:47.059769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:05:47.059769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:05:47.059769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:05:47.059769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:05:47.059769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:05:47.059769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:05:47.059769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:05:47.059769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:05:47.065091 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:05:47.065091 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:05:47.065091 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:05:47.065091 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:05:47.065091 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:05:47.065091 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:05:47.065091 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:05:47.065091 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:05:47.065091 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:05:47.065091 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:05:47.065091 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 06:05:47.065091 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:05:47.065091 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 109/215] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:05:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:05:47.065091 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358f690-93a2-4edc-b177-6a1723693cf8 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:47.085820 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3eaa6b-0648-4f3f-b1e3-eb1545683261 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:47.123399 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaa0d4e-381e-4d22-b957-9ac9fb5262cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:47.124204 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa0d4e-381e-4d22-b957-9ac9fb5262cb None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:47.124520 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa0d4e-381e-4d22-b957-9ac9fb5262cb None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:47.124804 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa0d4e-381e-4d22-b957-9ac9fb5262cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:47.126205 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaa0d4e-381e-4d22-b957-9ac9fb5262cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:47.127670 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaa0d4e-381e-4d22-b957-9ac9fb5262cb None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:47.129664 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 107/216] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:05:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:47.130247 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaa0d4e-381e-4d22-b957-9ac9fb5262cb None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:47.147134 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:47.170944 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358f690-93a2-4edc-b177-6a1723693cf8 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:47.201720 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:47.209290 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788d0c6-efcf-4825-a344-925629ebb502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:47.212495 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788d0c6-efcf-4825-a344-925629ebb502 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:47.212495 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788d0c6-efcf-4825-a344-925629ebb502 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:47.212495 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788d0c6-efcf-4825-a344-925629ebb502 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:47.222937 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9788d0c6-efcf-4825-a344-925629ebb502 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:05:47.230819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-9788d0c6-efcf-4825-a344-925629ebb502 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:05:47.230819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:05:47.230819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:05:47.230819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:05:47.230819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:05:47.230819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:05:47.230819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:05:47.230819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:05:47.230819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:05:47.230819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:05:47.230819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:05:47.230819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:05:47.230819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:05:47.230819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:05:47.230819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:05:47.230819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:05:47.231718 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:05:47.231718 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:05:47.231718 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:05:47.231718 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:05:47.231718 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:05:47.231718 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:05:47.231718 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:05:47.231718 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:05:47.231718 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:05:47.231718 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:05:47.231718 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:05:47.231718 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:05:47.231718 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:05:47.231718 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:05:47.231718 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:05:47.231718 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:05:47.232590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:05:47.232590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:05:47.232590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:05:47.232590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:05:47.232590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:05:47.232590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:05:47.232590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:05:47.232590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:05:47.232590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:05:47.232590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:05:47.232590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:05:47.232590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:05:47.232590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 110/217] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 18 06:05:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:05:47.233293 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788d0c6-efcf-4825-a344-925629ebb502 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:47.246769 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaa0d4e-381e-4d22-b957-9ac9fb5262cb None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:47.283015 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924f9c4b-5272-457e-8eea-6cf9b1543685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:47.283704 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f9c4b-5272-457e-8eea-6cf9b1543685 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:47.283997 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f9c4b-5272-457e-8eea-6cf9b1543685 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:47.284290 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f9c4b-5272-457e-8eea-6cf9b1543685 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:47.285087 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924f9c4b-5272-457e-8eea-6cf9b1543685 None admin] RBAC: Adding query filter params (name=member) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:47.285397 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924f9c4b-5272-457e-8eea-6cf9b1543685 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:47.286839 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924f9c4b-5272-457e-8eea-6cf9b1543685 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:47.296153 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 108/218] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 18 06:05:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:47.297573 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f3deb-5586-44d0-a118-4fa2316f819e None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:47.297869 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f3deb-5586-44d0-a118-4fa2316f819e None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:47.298154 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f3deb-5586-44d0-a118-4fa2316f819e None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:47.299550 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 109/219] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:05:47.300327 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f3deb-5586-44d0-a118-4fa2316f819e None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:47.344674 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788d0c6-efcf-4825-a344-925629ebb502 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:47.378294 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb497b-a3fb-46f7-9cba-14fec1ef528f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:47.378968 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb497b-a3fb-46f7-9cba-14fec1ef528f None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:47.379257 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb497b-a3fb-46f7-9cba-14fec1ef528f None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:47.379537 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb497b-a3fb-46f7-9cba-14fec1ef528f None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:47.380259 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eb497b-a3fb-46f7-9cba-14fec1ef528f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:47.381755 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eb497b-a3fb-46f7-9cba-14fec1ef528f None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:47.390815 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:47.405399 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f3deb-5586-44d0-a118-4fa2316f819e None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:47.431177 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:47.439884 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7b7bfc-4f29-4bec-8da6-304e5f5da563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:47.440594 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b7bfc-4f29-4bec-8da6-304e5f5da563 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:47.440885 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b7bfc-4f29-4bec-8da6-304e5f5da563 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:47.441168 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b7bfc-4f29-4bec-8da6-304e5f5da563 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:47.441917 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7b7bfc-4f29-4bec-8da6-304e5f5da563 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:47.442227 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7b7bfc-4f29-4bec-8da6-304e5f5da563 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:47.443765 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7b7bfc-4f29-4bec-8da6-304e5f5da563 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:47.452204 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 110/220] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:05:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:47.453167 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7b7bfc-4f29-4bec-8da6-304e5f5da563 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:47.482315 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 111/221] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Jul 18 06:05:47 2022] POST /identity/v3/users => generated 297 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:05:47.483648 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67145185-896b-4baa-bc8d-4e24d1260427 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:47.483942 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67145185-896b-4baa-bc8d-4e24d1260427 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:47.484442 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67145185-896b-4baa-bc8d-4e24d1260427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:47.502977 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-67145185-896b-4baa-bc8d-4e24d1260427 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:47.511510 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-67145185-896b-4baa-bc8d-4e24d1260427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:47.576696 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7b7bfc-4f29-4bec-8da6-304e5f5da563 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:47.580462 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 112/222] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:05:47.580462 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67145185-896b-4baa-bc8d-4e24d1260427 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:47.626380 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4dec23-f7b3-47ab-b402-ac32279508f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:47.627289 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4dec23-f7b3-47ab-b402-ac32279508f6 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:47.627628 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4dec23-f7b3-47ab-b402-ac32279508f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:47.627915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4dec23-f7b3-47ab-b402-ac32279508f6 None admin] PATH_INFO: `/v3/users/70ecfa9b1920435485830e0e7e41c2d8` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:47.643593 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4dec23-f7b3-47ab-b402-ac32279508f6 None admin] RBAC: Authorizing `identity:get_user(user_id=70ecfa9b1920435485830e0e7e41c2d8)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:47.645113 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4dec23-f7b3-47ab-b402-ac32279508f6 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:47.647021 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 111/223] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:05:47 2022] GET /identity/v3/users/70ecfa9b1920435485830e0e7e41c2d8 => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:47.647659 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4dec23-f7b3-47ab-b402-ac32279508f6 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:47.706015 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67145185-896b-4baa-bc8d-4e24d1260427 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:47.720185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:47.744855 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:47.756748 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b1b8c-9bc4-4cf0-9f3a-7dddb679fb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:47.757716 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b1b8c-9bc4-4cf0-9f3a-7dddb679fb34 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:47.757997 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b1b8c-9bc4-4cf0-9f3a-7dddb679fb34 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:47.758275 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b1b8c-9bc4-4cf0-9f3a-7dddb679fb34 None admin] PATH_INFO: `/v3/users/c44e70d11cad4202a68a87daf04ec41b` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:47.759772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3b1b8c-9bc4-4cf0-9f3a-7dddb679fb34 None admin] RBAC: Authorizing `identity:get_user(user_id=c44e70d11cad4202a68a87daf04ec41b)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:47.761571 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3b1b8c-9bc4-4cf0-9f3a-7dddb679fb34 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:47.763471 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 113/224] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:05:47 2022] GET /identity/v3/users/c44e70d11cad4202a68a87daf04ec41b => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:47.764801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b1b8c-9bc4-4cf0-9f3a-7dddb679fb34 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:47.790860 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4dec23-f7b3-47ab-b402-ac32279508f6 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:47.833585 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7adb97-18da-465f-8d3f-d5066d1ca88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:47.834519 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7adb97-18da-465f-8d3f-d5066d1ca88f None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:47.834788 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7adb97-18da-465f-8d3f-d5066d1ca88f None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:47.835068 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7adb97-18da-465f-8d3f-d5066d1ca88f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:47.849232 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7adb97-18da-465f-8d3f-d5066d1ca88f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:05:47.858836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-3d7adb97-18da-465f-8d3f-d5066d1ca88f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 06:05:47.858836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:05:47.858836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:05:47.858836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:05:47.858836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:05:47.858836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:05:47.858836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:05:47.858836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:05:47.858836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:05:47.858836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:05:47.858836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:05:47.858836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:05:47.858836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:05:47.858836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:05:47.858836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:05:47.858836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:05:47.859791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:05:47.859791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:05:47.859791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:05:47.859791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:05:47.859791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:05:47.859791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:05:47.859791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:05:47.859791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:05:47.859791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:05:47.859791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:05:47.859791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:05:47.859791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:05:47.859791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:05:47.859791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:05:47.859791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:05:47.859791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:05:47.860722 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:05:47.860722 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:05:47.860722 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:05:47.860722 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:05:47.860722 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:05:47.860722 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:05:47.860722 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:05:47.860722 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:05:47.860722 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:05:47.860722 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:05:47.860722 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 06:05:47.860722 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:05:47.860722 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 112/225] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 18 06:05:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:05:47.861380 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7adb97-18da-465f-8d3f-d5066d1ca88f None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:47.887850 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b1b8c-9bc4-4cf0-9f3a-7dddb679fb34 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:47.926815 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6afe7-c27c-4b9b-a8d7-13b9c0681842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:47.927881 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6afe7-c27c-4b9b-a8d7-13b9c0681842 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:47.928438 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6afe7-c27c-4b9b-a8d7-13b9c0681842 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:47.928914 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6afe7-c27c-4b9b-a8d7-13b9c0681842 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:47.995797 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7adb97-18da-465f-8d3f-d5066d1ca88f None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:48.005816 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:48.013910 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c6afe7-c27c-4b9b-a8d7-13b9c0681842 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:48.019855 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c6afe7-c27c-4b9b-a8d7-13b9c0681842 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:48.038725 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:48.039516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4967a-b6ff-4258-8212-3a1628a8fd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:48.040664 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4967a-b6ff-4258-8212-3a1628a8fd3c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:48.041024 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4967a-b6ff-4258-8212-3a1628a8fd3c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:48.041611 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4967a-b6ff-4258-8212-3a1628a8fd3c None admin] PATH_INFO: `/v3/projects/d34db96dd82a425d90973717021e4c95` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:48.043561 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d4967a-b6ff-4258-8212-3a1628a8fd3c None admin] RBAC: Authorizing `identity:get_project(project_id=d34db96dd82a425d90973717021e4c95)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:48.045586 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d4967a-b6ff-4258-8212-3a1628a8fd3c None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:48.052776 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 113/226] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:05:47 2022] GET /identity/v3/projects/d34db96dd82a425d90973717021e4c95 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:48.053717 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4967a-b6ff-4258-8212-3a1628a8fd3c None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:48.161312 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 114/227] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:05:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 06:05:48.163247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6afe7-c27c-4b9b-a8d7-13b9c0681842 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:48.169217 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4967a-b6ff-4258-8212-3a1628a8fd3c None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:48.208272 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a31f3a6-bc52-4e5e-bbfc-70e86b7e556e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:48.209111 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a31f3a6-bc52-4e5e-bbfc-70e86b7e556e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:48.209467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a31f3a6-bc52-4e5e-bbfc-70e86b7e556e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:48.209841 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a31f3a6-bc52-4e5e-bbfc-70e86b7e556e None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:48.210736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a31f3a6-bc52-4e5e-bbfc-70e86b7e556e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:48.211113 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a31f3a6-bc52-4e5e-bbfc-70e86b7e556e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:48.212671 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a31f3a6-bc52-4e5e-bbfc-70e86b7e556e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:48.221996 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 114/228] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:05:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:48.227318 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a31f3a6-bc52-4e5e-bbfc-70e86b7e556e None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:48.266029 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:48.274846 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:48.284654 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6afe7-c27c-4b9b-a8d7-13b9c0681842 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:48.325241 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe3e6dd-6bde-470c-b652-2e2878922677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:48.326575 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe3e6dd-6bde-470c-b652-2e2878922677 None admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:48.327019 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe3e6dd-6bde-470c-b652-2e2878922677 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:48.327476 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe3e6dd-6bde-470c-b652-2e2878922677 None admin] PATH_INFO: `/v3/projects/d34db96dd82a425d90973717021e4c95/users/c44e70d11cad4202a68a87daf04ec41b/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:48.329921 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe3e6dd-6bde-470c-b652-2e2878922677 None admin] RBAC: Authorizing `identity:create_grant(project_id=d34db96dd82a425d90973717021e4c95, user_id=c44e70d11cad4202a68a87daf04ec41b, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:48.331728 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe3e6dd-6bde-470c-b652-2e2878922677 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:48.341028 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a31f3a6-bc52-4e5e-bbfc-70e86b7e556e None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:48.383220 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe9776-62c1-4ac6-8f90-7a283d7bf113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:48.384101 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe9776-62c1-4ac6-8f90-7a283d7bf113 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:48.384388 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe9776-62c1-4ac6-8f90-7a283d7bf113 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:48.384690 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe9776-62c1-4ac6-8f90-7a283d7bf113 None admin] PATH_INFO: `/v3/projects/d34db96dd82a425d90973717021e4c95` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:48.385886 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fe9776-62c1-4ac6-8f90-7a283d7bf113 None admin] RBAC: Authorizing `identity:get_project(project_id=d34db96dd82a425d90973717021e4c95)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:48.387416 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fe9776-62c1-4ac6-8f90-7a283d7bf113 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:48.389559 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 115/229] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:05:48 2022] GET /identity/v3/projects/d34db96dd82a425d90973717021e4c95 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:48.390158 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe9776-62c1-4ac6-8f90-7a283d7bf113 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:48.438066 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 115/230] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 18 06:05:48 2022] PUT /identity/v3/projects/d34db96dd82a425d90973717021e4c95/users/c44e70d11cad4202a68a87daf04ec41b/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:05:48.439950 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe3e6dd-6bde-470c-b652-2e2878922677 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:48.587739 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe9776-62c1-4ac6-8f90-7a283d7bf113 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:48.597842 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:48.664906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe3e6dd-6bde-470c-b652-2e2878922677 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:48.665294 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e8d0e-9ffc-413e-b3d8-3dedcc1ee026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:48.666211 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e8d0e-9ffc-413e-b3d8-3dedcc1ee026 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:48.666501 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e8d0e-9ffc-413e-b3d8-3dedcc1ee026 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:48.666786 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e8d0e-9ffc-413e-b3d8-3dedcc1ee026 None admin] PATH_INFO: `/v3/users/c44e70d11cad4202a68a87daf04ec41b` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:48.670189 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747e8d0e-9ffc-413e-b3d8-3dedcc1ee026 None admin] RBAC: Authorizing `identity:get_user(user_id=c44e70d11cad4202a68a87daf04ec41b)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:48.671709 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747e8d0e-9ffc-413e-b3d8-3dedcc1ee026 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:48.673814 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 116/231] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:05:48 2022] GET /identity/v3/users/c44e70d11cad4202a68a87daf04ec41b => generated 283 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:48.680146 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e8d0e-9ffc-413e-b3d8-3dedcc1ee026 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:48.717816 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e52ff-38dd-44fb-b815-86f092e82c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:48.718530 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e52ff-38dd-44fb-b815-86f092e82c0f None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:48.718875 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e52ff-38dd-44fb-b815-86f092e82c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:48.719166 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e52ff-38dd-44fb-b815-86f092e82c0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:48.720158 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192e52ff-38dd-44fb-b815-86f092e82c0f None admin] RBAC: Adding query filter params (role.id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20, scope.project.id=d34db96dd82a425d90973717021e4c95, user.id=70ecfa9b1920435485830e0e7e41c2d8) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:48.720517 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192e52ff-38dd-44fb-b815-86f092e82c0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:48.722095 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192e52ff-38dd-44fb-b815-86f092e82c0f None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:48.728640 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 116/232] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:05:48 2022] GET /identity/v3/role_assignments?user.id=70ecfa9b1920435485830e0e7e41c2d8&scope.project.id=d34db96dd82a425d90973717021e4c95&role.id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 259 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:48.801725 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e8d0e-9ffc-413e-b3d8-3dedcc1ee026 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:48.846509 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8801104-b419-4178-b16a-0c756824cceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:48.847366 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8801104-b419-4178-b16a-0c756824cceb None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:48.847655 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8801104-b419-4178-b16a-0c756824cceb None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:48.847937 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8801104-b419-4178-b16a-0c756824cceb None admin] PATH_INFO: `/v3/projects/d34db96dd82a425d90973717021e4c95` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:48.849451 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8801104-b419-4178-b16a-0c756824cceb None admin] RBAC: Authorizing `identity:get_project(project_id=d34db96dd82a425d90973717021e4c95)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:48.851020 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8801104-b419-4178-b16a-0c756824cceb None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:48.853534 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 117/233] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:05:48 2022] GET /identity/v3/projects/d34db96dd82a425d90973717021e4c95 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:48.859928 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e52ff-38dd-44fb-b815-86f092e82c0f None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:48.890901 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:49.012676 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e52ff-38dd-44fb-b815-86f092e82c0f None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:49.053569 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a92f8e-c23a-4566-8852-cdff3fbc9f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:49.054482 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a92f8e-c23a-4566-8852-cdff3fbc9f8a None admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:49.054771 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a92f8e-c23a-4566-8852-cdff3fbc9f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:49.055050 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a92f8e-c23a-4566-8852-cdff3fbc9f8a None admin] PATH_INFO: `/v3/projects/d34db96dd82a425d90973717021e4c95/users/c44e70d11cad4202a68a87daf04ec41b/roles/04868303225144c389068779288f3a42` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:49.057383 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a92f8e-c23a-4566-8852-cdff3fbc9f8a None admin] RBAC: Authorizing `identity:create_grant(project_id=d34db96dd82a425d90973717021e4c95, user_id=c44e70d11cad4202a68a87daf04ec41b, role_id=04868303225144c389068779288f3a42)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:49.059421 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a92f8e-c23a-4566-8852-cdff3fbc9f8a None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:49.172783 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 117/234] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 18 06:05:48 2022] PUT /identity/v3/projects/d34db96dd82a425d90973717021e4c95/users/c44e70d11cad4202a68a87daf04ec41b/roles/04868303225144c389068779288f3a42 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:05:49.880223 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16b94e-18a0-4b20-ac0e-1af07e847852 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:49.880753 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16b94e-18a0-4b20-ac0e-1af07e847852 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:49.880856 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16b94e-18a0-4b20-ac0e-1af07e847852 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:49.882479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 118/235] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:05:49.888463 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c214a74-681e-4f09-88e6-136293b5ddaf None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:49.888894 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c214a74-681e-4f09-88e6-136293b5ddaf None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:49.889092 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c214a74-681e-4f09-88e6-136293b5ddaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:49.907103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-4c214a74-681e-4f09-88e6-136293b5ddaf None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:49.908297 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4c214a74-681e-4f09-88e6-136293b5ddaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:49.982388 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 118/236] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:05:50.006255 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c16b94e-18a0-4b20-ac0e-1af07e847852 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:50.062773 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:50.108469 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a841ef7-798f-47c7-9974-a0896f3e5a6c None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:50.109116 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a841ef7-798f-47c7-9974-a0896f3e5a6c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:50.109628 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a841ef7-798f-47c7-9974-a0896f3e5a6c None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:50.111233 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 119/237] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:05:50.116605 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e4b58-9cda-4f47-a784-b38148c74c42 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:50.117171 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e4b58-9cda-4f47-a784-b38148c74c42 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:50.117655 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e4b58-9cda-4f47-a784-b38148c74c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:50.130285 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:50.148441 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-615e4b58-9cda-4f47-a784-b38148c74c42 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:50.148441 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-615e4b58-9cda-4f47-a784-b38148c74c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:50.161260 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c16b94e-18a0-4b20-ac0e-1af07e847852 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:50.228296 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a92847-8c51-4ab5-8097-c651b4b76b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:50.229240 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a92847-8c51-4ab5-8097-c651b4b76b7a None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:50.229527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a92847-8c51-4ab5-8097-c651b4b76b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:50.230447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a92847-8c51-4ab5-8097-c651b4b76b7a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:50.247824 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a92847-8c51-4ab5-8097-c651b4b76b7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:05:50.266634 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 120/238] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 06:05:50.267569 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-d8a92847-8c51-4ab5-8097-c651b4b76b7a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 06:05:50.267569 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:05:50.267569 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:05:50.267569 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:05:50.267569 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:05:50.267569 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:05:50.267569 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:05:50.267569 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:05:50.267569 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:05:50.267569 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:05:50.267569 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:05:50.267569 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:05:50.267569 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:05:50.267569 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:05:50.267569 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:05:50.267569 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:05:50.268453 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:05:50.268453 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:05:50.268453 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:05:50.268453 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:05:50.268453 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:05:50.268453 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:05:50.268453 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:05:50.268453 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:05:50.268453 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:05:50.268453 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:05:50.268453 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:05:50.268453 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:05:50.268453 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:05:50.268453 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:05:50.268453 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:05:50.268453 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:05:50.269280 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:05:50.269280 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:05:50.269280 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:05:50.269280 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:05:50.269280 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:05:50.269280 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:05:50.269280 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:05:50.269280 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:05:50.269280 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:05:50.269280 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:05:50.269280 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 06:05:50.269280 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:05:50.269280 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 119/239] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:05:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:05:50.274991 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e4b58-9cda-4f47-a784-b38148c74c42 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:50.308526 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a92847-8c51-4ab5-8097-c651b4b76b7a None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:50.482908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e4b58-9cda-4f47-a784-b38148c74c42 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:50.517916 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:50.520711 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a92847-8c51-4ab5-8097-c651b4b76b7a None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:50.560917 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d7cf63-d1fd-4726-8fe3-7a879256de42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:50.561620 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eafdb1d-563a-4e65-a3bd-1bfac93cb483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:50.562276 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d7cf63-d1fd-4726-8fe3-7a879256de42 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:50.562767 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d7cf63-d1fd-4726-8fe3-7a879256de42 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:50.563238 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafdb1d-563a-4e65-a3bd-1bfac93cb483 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:50.563416 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d7cf63-d1fd-4726-8fe3-7a879256de42 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:50.563871 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafdb1d-563a-4e65-a3bd-1bfac93cb483 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:50.564631 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafdb1d-563a-4e65-a3bd-1bfac93cb483 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:50.565613 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eafdb1d-563a-4e65-a3bd-1bfac93cb483 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:50.566122 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eafdb1d-563a-4e65-a3bd-1bfac93cb483 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:50.567724 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eafdb1d-563a-4e65-a3bd-1bfac93cb483 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:50.569164 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d7cf63-d1fd-4726-8fe3-7a879256de42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:50.571252 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d7cf63-d1fd-4726-8fe3-7a879256de42 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:50.572981 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 120/240] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:05:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:05:50.579248 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:50.581894 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d7cf63-d1fd-4726-8fe3-7a879256de42 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:50.583516 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 121/241] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 18 06:05:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:50.597820 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eafdb1d-563a-4e65-a3bd-1bfac93cb483 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:50.786816 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d7cf63-d1fd-4726-8fe3-7a879256de42 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:50.791357 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eafdb1d-563a-4e65-a3bd-1bfac93cb483 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:50.856106 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a49ea-ba4e-45d3-833d-f31921df63d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:50.857369 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a49ea-ba4e-45d3-833d-f31921df63d9 None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:50.857665 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a49ea-ba4e-45d3-833d-f31921df63d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:50.857951 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a49ea-ba4e-45d3-833d-f31921df63d9 None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:50.858690 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71a49ea-ba4e-45d3-833d-f31921df63d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:50.860156 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71a49ea-ba4e-45d3-833d-f31921df63d9 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:50.862077 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b4b57f-c136-4eb7-8de1-1a179e3b83da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:50.862973 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4b57f-c136-4eb7-8de1-1a179e3b83da None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:50.863266 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4b57f-c136-4eb7-8de1-1a179e3b83da None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:50.863544 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4b57f-c136-4eb7-8de1-1a179e3b83da None admin] PATH_INFO: `/v3/users/74ac4fc2e1df423a86a4ee9c1137135a` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:50.886764 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b4b57f-c136-4eb7-8de1-1a179e3b83da None admin] RBAC: Authorizing `identity:get_user(user_id=74ac4fc2e1df423a86a4ee9c1137135a)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:50.888246 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b4b57f-c136-4eb7-8de1-1a179e3b83da None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:50.894816 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 122/242] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:05:50 2022] GET /identity/v3/users/74ac4fc2e1df423a86a4ee9c1137135a => generated 297 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:05:50.896592 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b4b57f-c136-4eb7-8de1-1a179e3b83da None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:50.952052 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 121/243] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Jul 18 06:05:50 2022] POST /identity/v3/users => generated 293 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:05:50.974625 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:51.027382 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9802418-9bcd-4855-9d8c-21bca6332503 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:51.028043 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9802418-9bcd-4855-9d8c-21bca6332503 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:51.028596 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9802418-9bcd-4855-9d8c-21bca6332503 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:51.033308 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 122/244] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:05:51.036275 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8294795-2614-4c38-bb6a-259c4d0e66f3 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:51.036948 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8294795-2614-4c38-bb6a-259c4d0e66f3 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:51.037440 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8294795-2614-4c38-bb6a-259c4d0e66f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:51.038431 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b4b57f-c136-4eb7-8de1-1a179e3b83da None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:51.072854 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-f8294795-2614-4c38-bb6a-259c4d0e66f3 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:51.075585 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-f8294795-2614-4c38-bb6a-259c4d0e66f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:51.078463 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a951f2-178f-4658-a616-5ae9b02e237c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:51.079588 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a951f2-178f-4658-a616-5ae9b02e237c None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:51.080066 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a951f2-178f-4658-a616-5ae9b02e237c None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:51.080557 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a951f2-178f-4658-a616-5ae9b02e237c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:51.120401 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:51.209639 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a951f2-178f-4658-a616-5ae9b02e237c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:51.211102 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a951f2-178f-4658-a616-5ae9b02e237c None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:51.230326 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 123/245] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:05:51.261881 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8294795-2614-4c38-bb6a-259c4d0e66f3 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:51.289315 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:51.356114 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 123/246] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:05:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 06:05:51.365709 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a951f2-178f-4658-a616-5ae9b02e237c None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:51.401128 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8294795-2614-4c38-bb6a-259c4d0e66f3 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:51.450885 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b631c-1627-420d-a347-94e5a14423be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:51.451701 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b631c-1627-420d-a347-94e5a14423be None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:51.451987 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b631c-1627-420d-a347-94e5a14423be None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:51.452262 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b631c-1627-420d-a347-94e5a14423be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:51.469122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5b631c-1627-420d-a347-94e5a14423be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:05:51.477224 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-0c5b631c-1627-420d-a347-94e5a14423be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:05:51.477224 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:05:51.477224 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:05:51.477224 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:05:51.477224 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:05:51.477224 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:05:51.477224 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:05:51.477224 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:05:51.477224 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:05:51.477224 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:05:51.477224 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:05:51.477224 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:05:51.477224 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:05:51.477224 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:05:51.477224 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:05:51.477224 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:05:51.478189 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:05:51.478189 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:05:51.478189 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:05:51.478189 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:05:51.478189 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:05:51.478189 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:05:51.478189 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:05:51.478189 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:05:51.478189 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:05:51.478189 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:05:51.478189 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:05:51.478189 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:05:51.478189 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:05:51.478189 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:05:51.478189 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:05:51.478189 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:05:51.479003 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:05:51.479003 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:05:51.479003 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:05:51.479003 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:05:51.479003 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:05:51.479003 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:05:51.479003 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:05:51.479003 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:05:51.479003 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:05:51.479003 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:05:51.479003 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:05:51.479003 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:05:51.482155 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 124/247] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 18 06:05:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:05:51.483756 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053227e0-a37b-489b-abfc-81f0b738adda None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:51.484211 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053227e0-a37b-489b-abfc-81f0b738adda None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:51.484747 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053227e0-a37b-489b-abfc-81f0b738adda None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:51.486496 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 125/248] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:05:51.489455 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053227e0-a37b-489b-abfc-81f0b738adda None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:51.510490 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a951f2-178f-4658-a616-5ae9b02e237c None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:51.511813 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:51.558690 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c27f5-706c-4e7b-8b26-c2f47acc4185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:51.559838 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c27f5-706c-4e7b-8b26-c2f47acc4185 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:51.560347 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c27f5-706c-4e7b-8b26-c2f47acc4185 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:51.560845 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c27f5-706c-4e7b-8b26-c2f47acc4185 None admin] PATH_INFO: `/v3/projects/d34db96dd82a425d90973717021e4c95` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:51.562755 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452c27f5-706c-4e7b-8b26-c2f47acc4185 None admin] RBAC: Authorizing `identity:get_project(project_id=d34db96dd82a425d90973717021e4c95)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:51.565216 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452c27f5-706c-4e7b-8b26-c2f47acc4185 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:51.567596 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 124/249] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:05:51 2022] GET /identity/v3/projects/d34db96dd82a425d90973717021e4c95 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:51.569348 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eadbbc-8b6c-471a-9087-875f33205456 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:51.569815 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eadbbc-8b6c-471a-9087-875f33205456 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:51.570260 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eadbbc-8b6c-471a-9087-875f33205456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:51.590915 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-00eadbbc-8b6c-471a-9087-875f33205456 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:51.592737 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-00eadbbc-8b6c-471a-9087-875f33205456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:51.629243 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:51.635184 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053227e0-a37b-489b-abfc-81f0b738adda None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:51.691829 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c172a99b-a238-4775-909f-ade3174ed91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:51.692897 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172a99b-a238-4775-909f-ade3174ed91c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:51.693362 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172a99b-a238-4775-909f-ade3174ed91c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:51.694460 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172a99b-a238-4775-909f-ade3174ed91c None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:51.695645 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c172a99b-a238-4775-909f-ade3174ed91c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:51.696090 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c172a99b-a238-4775-909f-ade3174ed91c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:51.698216 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c172a99b-a238-4775-909f-ade3174ed91c None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:51.717818 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 126/251] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:05:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:51.718006 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 125/251] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:05:51.721937 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eadbbc-8b6c-471a-9087-875f33205456 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:51.725315 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c172a99b-a238-4775-909f-ade3174ed91c None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:51.852198 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:51.873437 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c172a99b-a238-4775-909f-ade3174ed91c None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:51.904632 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eadbbc-8b6c-471a-9087-875f33205456 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:51.919280 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79785ee-f83e-434a-b73b-15280bd00834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:51.920850 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79785ee-f83e-434a-b73b-15280bd00834 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:51.921339 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79785ee-f83e-434a-b73b-15280bd00834 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:51.921886 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79785ee-f83e-434a-b73b-15280bd00834 None admin] PATH_INFO: `/v3/users/c44e70d11cad4202a68a87daf04ec41b` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:51.923922 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79785ee-f83e-434a-b73b-15280bd00834 None admin] RBAC: Authorizing `identity:get_user(user_id=c44e70d11cad4202a68a87daf04ec41b)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:51.926081 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79785ee-f83e-434a-b73b-15280bd00834 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:51.933752 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 127/252] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:05:51 2022] GET /identity/v3/users/c44e70d11cad4202a68a87daf04ec41b => generated 283 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:51.935399 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79785ee-f83e-434a-b73b-15280bd00834 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:51.978019 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc6d1be-9db1-4484-9570-726b8a27d4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:51.979115 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6d1be-9db1-4484-9570-726b8a27d4d9 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:51.979621 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6d1be-9db1-4484-9570-726b8a27d4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:51.980783 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6d1be-9db1-4484-9570-726b8a27d4d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:51.981936 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc6d1be-9db1-4484-9570-726b8a27d4d9 None admin] RBAC: Adding query filter params (role.id=432bfb33f0224d11acc5bae2c196dc77, scope.project.id=d34db96dd82a425d90973717021e4c95, user.id=74ac4fc2e1df423a86a4ee9c1137135a) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:51.982460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc6d1be-9db1-4484-9570-726b8a27d4d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:51.984178 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc6d1be-9db1-4484-9570-726b8a27d4d9 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:51.990792 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 126/253] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:05:51 2022] GET /identity/v3/role_assignments?user.id=74ac4fc2e1df423a86a4ee9c1137135a&scope.project.id=d34db96dd82a425d90973717021e4c95&role.id=432bfb33f0224d11acc5bae2c196dc77 => generated 259 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:51.991954 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc6d1be-9db1-4484-9570-726b8a27d4d9 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:52.015756 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:52.088465 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79785ee-f83e-434a-b73b-15280bd00834 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:52.137177 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc6d1be-9db1-4484-9570-726b8a27d4d9 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:52.161561 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db25b63-96da-4f3f-bee1-16b6a51d3575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:52.164302 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db25b63-96da-4f3f-bee1-16b6a51d3575 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:52.164888 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db25b63-96da-4f3f-bee1-16b6a51d3575 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:52.165370 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db25b63-96da-4f3f-bee1-16b6a51d3575 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:52.190435 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db25b63-96da-4f3f-bee1-16b6a51d3575 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:05:52.203267 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-9db25b63-96da-4f3f-bee1-16b6a51d3575 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 06:05:52.203267 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:05:52.203267 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:05:52.203267 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:05:52.203267 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:05:52.203267 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:05:52.203267 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:05:52.203267 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:05:52.203267 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:05:52.203267 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:05:52.203267 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:05:52.203267 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:05:52.203267 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:05:52.203267 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:05:52.203267 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:05:52.203267 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:05:52.204234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:05:52.204234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:05:52.204234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:05:52.204234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:05:52.204234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:05:52.204234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:05:52.204234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:05:52.204234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:05:52.204234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:05:52.204234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:05:52.204234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:05:52.204234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:05:52.204234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:05:52.204234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:05:52.204234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:05:52.204234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:05:52.205057 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:05:52.205057 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:05:52.205057 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:05:52.205057 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:05:52.205057 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:05:52.205057 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:05:52.205057 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:05:52.205057 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:05:52.205057 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:05:52.205057 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:05:52.205057 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 06:05:52.205057 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:05:52.210807 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 128/254] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:05:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:05:52.215060 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da72fde-a879-46d7-b1a7-8113bb879ee9 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:52.215631 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da72fde-a879-46d7-b1a7-8113bb879ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:52.216732 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da72fde-a879-46d7-b1a7-8113bb879ee9 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:52.219778 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f71c6d-d846-4423-9a96-cdfe156d35e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:52.220797 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f71c6d-d846-4423-9a96-cdfe156d35e0 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:52.221140 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 129/255] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:52 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:05:52.221511 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f71c6d-d846-4423-9a96-cdfe156d35e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:52.221966 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da72fde-a879-46d7-b1a7-8113bb879ee9 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:52.222198 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f71c6d-d846-4423-9a96-cdfe156d35e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:52.231856 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:52.322339 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f71c6d-d846-4423-9a96-cdfe156d35e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:52.324563 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f71c6d-d846-4423-9a96-cdfe156d35e0 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:52.348385 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:52.451483 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 127/256] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:05:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 06:05:52.452916 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053274ea-ccfa-44f5-9b4a-023cd2864a80 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:52.453234 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053274ea-ccfa-44f5-9b4a-023cd2864a80 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:52.453520 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053274ea-ccfa-44f5-9b4a-023cd2864a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:52.472445 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-053274ea-ccfa-44f5-9b4a-023cd2864a80 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:52.473641 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-053274ea-ccfa-44f5-9b4a-023cd2864a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:52.541114 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da72fde-a879-46d7-b1a7-8113bb879ee9 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:52.548178 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 128/257] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:05:52.549090 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053274ea-ccfa-44f5-9b4a-023cd2864a80 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:52.627542 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:52.686521 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053274ea-ccfa-44f5-9b4a-023cd2864a80 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:52.705231 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ce9f5-77be-45e8-b2eb-3f9b5c18f882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:52.705853 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ce9f5-77be-45e8-b2eb-3f9b5c18f882 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:52.706141 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ce9f5-77be-45e8-b2eb-3f9b5c18f882 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:52.706414 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ce9f5-77be-45e8-b2eb-3f9b5c18f882 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:52.707181 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69ce9f5-77be-45e8-b2eb-3f9b5c18f882 None admin] RBAC: Adding query filter params (name=member) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:52.707490 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69ce9f5-77be-45e8-b2eb-3f9b5c18f882 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:52.708929 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69ce9f5-77be-45e8-b2eb-3f9b5c18f882 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:52.724381 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e607b4aa-3143-45f2-a94d-308999b4a202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:52.725305 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e607b4aa-3143-45f2-a94d-308999b4a202 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:52.725591 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e607b4aa-3143-45f2-a94d-308999b4a202 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:52.725868 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e607b4aa-3143-45f2-a94d-308999b4a202 None admin] PATH_INFO: `/v3/projects/d34db96dd82a425d90973717021e4c95` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:52.727078 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e607b4aa-3143-45f2-a94d-308999b4a202 None admin] RBAC: Authorizing `identity:get_project(project_id=d34db96dd82a425d90973717021e4c95)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:52.728598 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e607b4aa-3143-45f2-a94d-308999b4a202 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:52.731076 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 129/258] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:05:52 2022] GET /identity/v3/projects/d34db96dd82a425d90973717021e4c95 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:52.731874 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e607b4aa-3143-45f2-a94d-308999b4a202 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:52.739015 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 130/259] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 18 06:05:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 517 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:52.739883 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ce9f5-77be-45e8-b2eb-3f9b5c18f882 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:52.845741 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e607b4aa-3143-45f2-a94d-308999b4a202 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:52.869884 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:52.883758 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b729ae9-deb2-4ad5-b5fd-75212cf875a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:52.884663 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b729ae9-deb2-4ad5-b5fd-75212cf875a1 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:52.884954 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b729ae9-deb2-4ad5-b5fd-75212cf875a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:52.885235 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b729ae9-deb2-4ad5-b5fd-75212cf875a1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:52.905117 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b729ae9-deb2-4ad5-b5fd-75212cf875a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:05:52.906339 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:52.925277 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-2b729ae9-deb2-4ad5-b5fd-75212cf875a1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 06:05:52.925277 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:05:52.925277 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:05:52.925277 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:05:52.925277 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:05:52.925277 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:05:52.925277 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:05:52.925277 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:05:52.925277 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:05:52.925277 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:05:52.925277 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:05:52.925277 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:05:52.925277 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:05:52.925277 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:05:52.925277 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:05:52.925277 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:05:52.927420 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:05:52.927420 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:05:52.927420 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:05:52.927420 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:05:52.927420 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:05:52.927420 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:05:52.927420 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:05:52.927420 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:05:52.927420 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:05:52.927420 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:05:52.927420 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:05:52.927420 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:05:52.927420 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:05:52.927420 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:05:52.927420 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:05:52.927420 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:05:52.928199 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:05:52.928199 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:05:52.928199 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:05:52.928199 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:05:52.928199 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:05:52.928199 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:05:52.928199 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:05:52.928199 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:05:52.928199 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:05:52.928199 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:05:52.928199 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 06:05:52.928199 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:05:52.928199 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 130/260] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 18 06:05:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:05:52.928199 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b729ae9-deb2-4ad5-b5fd-75212cf875a1 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:52.948018 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ce9f5-77be-45e8-b2eb-3f9b5c18f882 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:52.986568 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db4e8c-9004-4966-98a3-d05bf517f7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:52.987583 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db4e8c-9004-4966-98a3-d05bf517f7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:52.988089 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db4e8c-9004-4966-98a3-d05bf517f7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:52.988804 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db4e8c-9004-4966-98a3-d05bf517f7e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:52.990324 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db4e8c-9004-4966-98a3-d05bf517f7e0 None admin] RBAC: Adding query filter params (role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc, scope.project.id=d34db96dd82a425d90973717021e4c95, user.id=c44e70d11cad4202a68a87daf04ec41b) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:52.990979 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db4e8c-9004-4966-98a3-d05bf517f7e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:52.992856 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db4e8c-9004-4966-98a3-d05bf517f7e0 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:53.000557 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 131/261] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:05:52 2022] GET /identity/v3/role_assignments?user.id=c44e70d11cad4202a68a87daf04ec41b&scope.project.id=d34db96dd82a425d90973717021e4c95&role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 608 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:53.002167 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db4e8c-9004-4966-98a3-d05bf517f7e0 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:53.170168 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db4e8c-9004-4966-98a3-d05bf517f7e0 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:53.178824 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b729ae9-deb2-4ad5-b5fd-75212cf875a1 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:53.211432 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76426eba-531b-46df-bd99-755b39858eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:53.212610 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76426eba-531b-46df-bd99-755b39858eea None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:53.213100 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76426eba-531b-46df-bd99-755b39858eea None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:53.213579 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76426eba-531b-46df-bd99-755b39858eea None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:53.228928 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:53.234307 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76426eba-531b-46df-bd99-755b39858eea None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:53.234624 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76426eba-531b-46df-bd99-755b39858eea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:53.236089 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76426eba-531b-46df-bd99-755b39858eea None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:53.255572 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710b4af-87d5-47b8-a567-1c249acb0c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:53.256465 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710b4af-87d5-47b8-a567-1c249acb0c58 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:53.256762 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710b4af-87d5-47b8-a567-1c249acb0c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:53.257042 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710b4af-87d5-47b8-a567-1c249acb0c58 None admin] PATH_INFO: `/v3/users/70ecfa9b1920435485830e0e7e41c2d8` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:53.262449 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a710b4af-87d5-47b8-a567-1c249acb0c58 None admin] RBAC: Authorizing `identity:get_user(user_id=70ecfa9b1920435485830e0e7e41c2d8)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:53.264015 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a710b4af-87d5-47b8-a567-1c249acb0c58 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:53.266297 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 131/262] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:05:52 2022] GET /identity/v3/users/70ecfa9b1920435485830e0e7e41c2d8 => generated 297 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:53.266480 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 132/263] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:05:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:53.271849 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710b4af-87d5-47b8-a567-1c249acb0c58 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:53.281537 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76426eba-531b-46df-bd99-755b39858eea None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:53.386832 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:53.450422 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76426eba-531b-46df-bd99-755b39858eea None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:53.482646 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710b4af-87d5-47b8-a567-1c249acb0c58 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:53.488903 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e092f8c-6906-4370-be0b-fe8a9a1bc001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:53.489875 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e092f8c-6906-4370-be0b-fe8a9a1bc001 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:53.490172 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e092f8c-6906-4370-be0b-fe8a9a1bc001 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:53.490455 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e092f8c-6906-4370-be0b-fe8a9a1bc001 None admin] PATH_INFO: `/v3/users/c44e70d11cad4202a68a87daf04ec41b` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:53.491809 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e092f8c-6906-4370-be0b-fe8a9a1bc001 None admin] RBAC: Authorizing `identity:get_user(user_id=c44e70d11cad4202a68a87daf04ec41b)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:53.493528 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e092f8c-6906-4370-be0b-fe8a9a1bc001 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:53.497244 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 133/264] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:05:53 2022] GET /identity/v3/users/c44e70d11cad4202a68a87daf04ec41b => generated 283 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:53.503327 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e092f8c-6906-4370-be0b-fe8a9a1bc001 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:53.550356 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f66ef-2d22-42d8-9f54-5108fb488154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:53.551168 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f66ef-2d22-42d8-9f54-5108fb488154 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:53.551451 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f66ef-2d22-42d8-9f54-5108fb488154 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:53.551724 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f66ef-2d22-42d8-9f54-5108fb488154 None admin] PATH_INFO: `/v3/projects/d34db96dd82a425d90973717021e4c95` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:53.557202 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070f66ef-2d22-42d8-9f54-5108fb488154 None admin] RBAC: Authorizing `identity:get_project(project_id=d34db96dd82a425d90973717021e4c95)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:53.559196 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070f66ef-2d22-42d8-9f54-5108fb488154 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:53.561310 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 132/265] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:05:53 2022] GET /identity/v3/projects/d34db96dd82a425d90973717021e4c95 => generated 302 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:53.567609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f66ef-2d22-42d8-9f54-5108fb488154 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:53.574123 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:53.661960 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e092f8c-6906-4370-be0b-fe8a9a1bc001 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:53.739649 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bddbba-05b8-4002-8d68-19d7a2da2ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:53.740334 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bddbba-05b8-4002-8d68-19d7a2da2ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:53.740646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bddbba-05b8-4002-8d68-19d7a2da2ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:53.740926 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bddbba-05b8-4002-8d68-19d7a2da2ad3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:53.791201 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f66ef-2d22-42d8-9f54-5108fb488154 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:53.810809 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:53.832157 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bddbba-05b8-4002-8d68-19d7a2da2ad3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:53.833695 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bddbba-05b8-4002-8d68-19d7a2da2ad3 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:53.855807 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b953c520-cfeb-4cde-927a-b73136ccce51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:53.856844 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953c520-cfeb-4cde-927a-b73136ccce51 None admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:53.857137 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953c520-cfeb-4cde-927a-b73136ccce51 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:53.857422 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953c520-cfeb-4cde-927a-b73136ccce51 None admin] PATH_INFO: `/v3/projects/d34db96dd82a425d90973717021e4c95/users/70ecfa9b1920435485830e0e7e41c2d8/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:53.862826 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b953c520-cfeb-4cde-927a-b73136ccce51 None admin] RBAC: Authorizing `identity:create_grant(project_id=d34db96dd82a425d90973717021e4c95, user_id=70ecfa9b1920435485830e0e7e41c2d8, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:53.864486 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b953c520-cfeb-4cde-927a-b73136ccce51 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:53.901248 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:53.946328 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 134/266] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:05:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 06:05:53.952457 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bddbba-05b8-4002-8d68-19d7a2da2ad3 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:53.995138 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 133/267] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 18 06:05:53 2022] PUT /identity/v3/projects/d34db96dd82a425d90973717021e4c95/users/70ecfa9b1920435485830e0e7e41c2d8/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:05:54.052764 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db00f3-4979-474b-9f0b-f523e1291f08 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:54.053075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db00f3-4979-474b-9f0b-f523e1291f08 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:54.053333 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db00f3-4979-474b-9f0b-f523e1291f08 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:54.054966 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 134/268] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:05:54.057512 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316acbce-ec07-4244-865c-d7fdfb97e9e0 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:54.058422 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316acbce-ec07-4244-865c-d7fdfb97e9e0 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:54.058634 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316acbce-ec07-4244-865c-d7fdfb97e9e0 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:54.060467 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 135/269] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:05:54.065204 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b534f5-feb4-4a86-b7a1-593ab27ce8af None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:54.065421 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b534f5-feb4-4a86-b7a1-593ab27ce8af None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:54.065704 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b534f5-feb4-4a86-b7a1-593ab27ce8af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:54.084686 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-08b534f5-feb4-4a86-b7a1-593ab27ce8af None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:54.085996 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-08b534f5-feb4-4a86-b7a1-593ab27ce8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:54.149153 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bddbba-05b8-4002-8d68-19d7a2da2ad3 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:54.172585 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 136/270] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 06:05:54.174248 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247332d-b47a-4997-9021-a4ffcace9411 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:54.174650 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247332d-b47a-4997-9021-a4ffcace9411 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:54.174939 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247332d-b47a-4997-9021-a4ffcace9411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:54.198545 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-1247332d-b47a-4997-9021-a4ffcace9411 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:54.199763 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-1247332d-b47a-4997-9021-a4ffcace9411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:54.222584 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad7807-7018-46a3-9e29-746f0fdefedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:54.224245 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad7807-7018-46a3-9e29-746f0fdefedb None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:54.224814 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad7807-7018-46a3-9e29-746f0fdefedb None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:54.225324 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad7807-7018-46a3-9e29-746f0fdefedb None admin] PATH_INFO: `/v3/projects/d34db96dd82a425d90973717021e4c95` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:54.227457 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad7807-7018-46a3-9e29-746f0fdefedb None admin] RBAC: Authorizing `identity:get_project(project_id=d34db96dd82a425d90973717021e4c95)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:54.231095 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad7807-7018-46a3-9e29-746f0fdefedb None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:54.234410 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 135/271] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:05:53 2022] GET /identity/v3/projects/d34db96dd82a425d90973717021e4c95 => generated 302 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:54.235783 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad7807-7018-46a3-9e29-746f0fdefedb None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:54.272514 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:54.290087 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 137/272] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:05:54.292854 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1247332d-b47a-4997-9021-a4ffcace9411 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:54.337806 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:54.367394 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad7807-7018-46a3-9e29-746f0fdefedb None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:54.404452 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed62a77-c7d4-4010-99d2-f580038843cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:54.405633 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed62a77-c7d4-4010-99d2-f580038843cd None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:54.406125 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed62a77-c7d4-4010-99d2-f580038843cd None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:54.406632 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed62a77-c7d4-4010-99d2-f580038843cd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:54.408079 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed62a77-c7d4-4010-99d2-f580038843cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:54.409821 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed62a77-c7d4-4010-99d2-f580038843cd None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:54.411709 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 136/273] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:05:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:54.412966 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed62a77-c7d4-4010-99d2-f580038843cd None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:54.419365 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1247332d-b47a-4997-9021-a4ffcace9411 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:54.459233 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f3c6e-cab1-497b-a81a-46ff726d9ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:54.459897 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f3c6e-cab1-497b-a81a-46ff726d9ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:54.460238 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f3c6e-cab1-497b-a81a-46ff726d9ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:54.460540 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f3c6e-cab1-497b-a81a-46ff726d9ae7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:54.461327 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034f3c6e-cab1-497b-a81a-46ff726d9ae7 None admin] RBAC: Adding query filter params (role.id=04868303225144c389068779288f3a42, scope.project.id=d34db96dd82a425d90973717021e4c95, user.id=c44e70d11cad4202a68a87daf04ec41b) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:54.461642 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034f3c6e-cab1-497b-a81a-46ff726d9ae7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:54.463100 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034f3c6e-cab1-497b-a81a-46ff726d9ae7 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:54.472590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 138/274] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:05:54 2022] GET /identity/v3/role_assignments?user.id=c44e70d11cad4202a68a87daf04ec41b&scope.project.id=d34db96dd82a425d90973717021e4c95&role.id=04868303225144c389068779288f3a42 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:54.472590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f3c6e-cab1-497b-a81a-46ff726d9ae7 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:54.548941 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:54.553151 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed62a77-c7d4-4010-99d2-f580038843cd None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:54.609995 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378dd2b5-b380-4c60-ba3f-c4ae8b1b0c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:54.610866 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dd2b5-b380-4c60-ba3f-c4ae8b1b0c99 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:54.611203 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dd2b5-b380-4c60-ba3f-c4ae8b1b0c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:54.611680 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dd2b5-b380-4c60-ba3f-c4ae8b1b0c99 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:54.616023 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:54.625982 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f3c6e-cab1-497b-a81a-46ff726d9ae7 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:54.636903 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378dd2b5-b380-4c60-ba3f-c4ae8b1b0c99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:05:54.637035 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-378dd2b5-b380-4c60-ba3f-c4ae8b1b0c99 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 06:05:54.637035 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:05:54.637035 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:05:54.637035 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:05:54.637035 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:05:54.637035 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:05:54.637035 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:05:54.637035 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:05:54.637035 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:05:54.637035 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:05:54.637035 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:05:54.637035 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:05:54.637035 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:05:54.637035 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:05:54.637035 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:05:54.637035 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:05:54.637882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:05:54.637882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:05:54.637882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:05:54.637882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:05:54.637882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:05:54.637882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:05:54.637882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:05:54.637882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:05:54.637882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:05:54.637882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:05:54.637882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:05:54.637882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:05:54.637882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:05:54.637882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:05:54.637882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:05:54.637882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:05:54.638682 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:05:54.638682 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:05:54.638682 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:05:54.638682 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:05:54.638682 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:05:54.638682 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:05:54.638682 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:05:54.638682 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:05:54.638682 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:05:54.638682 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:05:54.638682 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 06:05:54.638682 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:05:54.638682 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 137/275] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:05:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:05:54.640122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eaabd6-5f0a-44d1-9ad8-f4d83e1acd24 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:54.640122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eaabd6-5f0a-44d1-9ad8-f4d83e1acd24 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:54.640122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eaabd6-5f0a-44d1-9ad8-f4d83e1acd24 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:54.641789 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 138/276] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:05:54.647275 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de932fb-45dd-4bcb-8142-2afaf9b968ce None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:54.647536 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de932fb-45dd-4bcb-8142-2afaf9b968ce None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:54.647820 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de932fb-45dd-4bcb-8142-2afaf9b968ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:54.675899 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-6de932fb-45dd-4bcb-8142-2afaf9b968ce None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:54.678198 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-6de932fb-45dd-4bcb-8142-2afaf9b968ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:54.679708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b7b465-dd10-4395-9427-b90e26033ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:54.680490 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7b465-dd10-4395-9427-b90e26033ab5 None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:54.680751 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7b465-dd10-4395-9427-b90e26033ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:54.681115 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7b465-dd10-4395-9427-b90e26033ab5 None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:54.686140 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b7b465-dd10-4395-9427-b90e26033ab5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:54.686140 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b7b465-dd10-4395-9427-b90e26033ab5 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:54.772229 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 139/277] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 06:05:54.780616 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de932fb-45dd-4bcb-8142-2afaf9b968ce None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:54.795042 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 139/278] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Jul 18 06:05:54 2022] POST /identity/v3/users => generated 293 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:05:54.820720 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b7b465-dd10-4395-9427-b90e26033ab5 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:54.897680 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:54.922725 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de932fb-45dd-4bcb-8142-2afaf9b968ce None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:54.978518 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b7b465-dd10-4395-9427-b90e26033ab5 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:54.997135 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ddd03b-c6ff-497c-8e5f-08686d1769e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:54.997897 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ddd03b-c6ff-497c-8e5f-08686d1769e7 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:54.998166 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ddd03b-c6ff-497c-8e5f-08686d1769e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:54.998459 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ddd03b-c6ff-497c-8e5f-08686d1769e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:54.999233 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ddd03b-c6ff-497c-8e5f-08686d1769e7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:54.999546 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ddd03b-c6ff-497c-8e5f-08686d1769e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:55.002114 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ddd03b-c6ff-497c-8e5f-08686d1769e7 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:55.010513 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 140/279] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 18 06:05:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:55.017280 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ddd03b-c6ff-497c-8e5f-08686d1769e7 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:55.019513 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73beffb-87b7-4f15-abc9-d341b2f05846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:55.020337 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73beffb-87b7-4f15-abc9-d341b2f05846 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:55.020666 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73beffb-87b7-4f15-abc9-d341b2f05846 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:55.020943 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73beffb-87b7-4f15-abc9-d341b2f05846 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:55.035958 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73beffb-87b7-4f15-abc9-d341b2f05846 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:05:55.046383 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-f73beffb-87b7-4f15-abc9-d341b2f05846 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 06:05:55.046383 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:05:55.046383 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:05:55.046383 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:05:55.046383 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:05:55.046383 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:05:55.046383 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:05:55.046383 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:05:55.046383 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:05:55.046383 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:05:55.046383 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:05:55.046383 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:05:55.046383 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:05:55.046383 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:05:55.046383 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:05:55.046383 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:05:55.047343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:05:55.047343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:05:55.047343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:05:55.047343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:05:55.047343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:05:55.047343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:05:55.047343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:05:55.047343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:05:55.047343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:05:55.047343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:05:55.047343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:05:55.047343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:05:55.047343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:05:55.047343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:05:55.047343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:05:55.047343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:05:55.048116 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:05:55.048116 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:05:55.048116 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:05:55.048116 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:05:55.048116 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:05:55.048116 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:05:55.048116 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:05:55.048116 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:05:55.048116 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:05:55.048116 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:05:55.048116 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 06:05:55.048116 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:05:55.049022 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 140/280] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:05:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:05:55.054301 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73beffb-87b7-4f15-abc9-d341b2f05846 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:55.072928 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:55.171980 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ddd03b-c6ff-497c-8e5f-08686d1769e7 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:55.201714 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73beffb-87b7-4f15-abc9-d341b2f05846 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:55.216882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aa21ab-4357-4739-a219-2da0f7ccafc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:55.217788 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa21ab-4357-4739-a219-2da0f7ccafc0 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:55.218071 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa21ab-4357-4739-a219-2da0f7ccafc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:55.218395 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa21ab-4357-4739-a219-2da0f7ccafc0 None admin] PATH_INFO: `/v3/users/6228fb05d5d74f6babe66d0649479859` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:55.241770 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aa21ab-4357-4739-a219-2da0f7ccafc0 None admin] RBAC: Authorizing `identity:get_user(user_id=6228fb05d5d74f6babe66d0649479859)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:55.243540 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aa21ab-4357-4739-a219-2da0f7ccafc0 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:55.246875 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 141/281] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:05:55 2022] GET /identity/v3/users/6228fb05d5d74f6babe66d0649479859 => generated 293 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:05:55.252829 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aa21ab-4357-4739-a219-2da0f7ccafc0 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:55.253261 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd634b9f-cc2c-434f-93db-520ba8cc86a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:55.254450 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd634b9f-cc2c-434f-93db-520ba8cc86a8 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:55.254943 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd634b9f-cc2c-434f-93db-520ba8cc86a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:55.255421 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd634b9f-cc2c-434f-93db-520ba8cc86a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:55.256497 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd634b9f-cc2c-434f-93db-520ba8cc86a8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:55.256822 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd634b9f-cc2c-434f-93db-520ba8cc86a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:55.259320 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd634b9f-cc2c-434f-93db-520ba8cc86a8 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:55.265181 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:55.271596 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 141/282] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 18 06:05:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:05:55.277852 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd634b9f-cc2c-434f-93db-520ba8cc86a8 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:55.427912 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:55.452559 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aa21ab-4357-4739-a219-2da0f7ccafc0 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:55.498689 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd634b9f-cc2c-434f-93db-520ba8cc86a8 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:55.537349 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4144cec-47e7-46bc-b1bc-a62672d7692f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:55.538675 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4144cec-47e7-46bc-b1bc-a62672d7692f None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:55.539160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4144cec-47e7-46bc-b1bc-a62672d7692f None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:55.539635 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4144cec-47e7-46bc-b1bc-a62672d7692f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:55.540780 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4144cec-47e7-46bc-b1bc-a62672d7692f None admin] RBAC: Adding query filter params (role.id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20, user.id=6228fb05d5d74f6babe66d0649479859, scope.system=all) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:55.541335 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4144cec-47e7-46bc-b1bc-a62672d7692f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:55.543051 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4144cec-47e7-46bc-b1bc-a62672d7692f None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:55.546256 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487fe0c-2675-4c69-ae68-44b525f84884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:55.547151 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487fe0c-2675-4c69-ae68-44b525f84884 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:55.547432 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487fe0c-2675-4c69-ae68-44b525f84884 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:55.547706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487fe0c-2675-4c69-ae68-44b525f84884 None admin] PATH_INFO: `/v3/users/74ac4fc2e1df423a86a4ee9c1137135a` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:55.548983 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1487fe0c-2675-4c69-ae68-44b525f84884 None admin] RBAC: Authorizing `identity:get_user(user_id=74ac4fc2e1df423a86a4ee9c1137135a)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:55.550598 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1487fe0c-2675-4c69-ae68-44b525f84884 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:55.552845 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 142/283] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:05:55 2022] GET /identity/v3/users/74ac4fc2e1df423a86a4ee9c1137135a => generated 297 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:05:55.554949 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 142/284] 10.222.0.40 () {56 vars in 1308 bytes} [Mon Jul 18 06:05:55 2022] GET /identity/v3/role_assignments?user.id=6228fb05d5d74f6babe66d0649479859&scope.system=all&role.id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 226 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:55.563126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487fe0c-2675-4c69-ae68-44b525f84884 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:55.740614 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487fe0c-2675-4c69-ae68-44b525f84884 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:55.803701 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77462052-e145-4106-ab68-4db0c0343a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:55.804620 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77462052-e145-4106-ab68-4db0c0343a01 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:55.804912 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77462052-e145-4106-ab68-4db0c0343a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:55.805193 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77462052-e145-4106-ab68-4db0c0343a01 None admin] PATH_INFO: `/v3/projects/d34db96dd82a425d90973717021e4c95` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:55.806651 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77462052-e145-4106-ab68-4db0c0343a01 None admin] RBAC: Authorizing `identity:get_project(project_id=d34db96dd82a425d90973717021e4c95)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:55.808454 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77462052-e145-4106-ab68-4db0c0343a01 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:55.811668 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 143/285] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:05:55 2022] GET /identity/v3/projects/d34db96dd82a425d90973717021e4c95 => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:55.816098 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4144cec-47e7-46bc-b1bc-a62672d7692f None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:55.915725 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:55.958849 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4144cec-47e7-46bc-b1bc-a62672d7692f None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:56.013366 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f668613-6d3d-47de-98d3-8f4f77bd5d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:56.014743 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f668613-6d3d-47de-98d3-8f4f77bd5d83 None admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:56.015227 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f668613-6d3d-47de-98d3-8f4f77bd5d83 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:56.015706 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f668613-6d3d-47de-98d3-8f4f77bd5d83 None admin] PATH_INFO: `/v3/projects/d34db96dd82a425d90973717021e4c95/users/74ac4fc2e1df423a86a4ee9c1137135a/roles/432bfb33f0224d11acc5bae2c196dc77` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:56.022806 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f668613-6d3d-47de-98d3-8f4f77bd5d83 None admin] RBAC: Authorizing `identity:create_grant(project_id=d34db96dd82a425d90973717021e4c95, user_id=74ac4fc2e1df423a86a4ee9c1137135a, role_id=432bfb33f0224d11acc5bae2c196dc77)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:56.025309 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f668613-6d3d-47de-98d3-8f4f77bd5d83 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:56.110043 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 143/286] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 18 06:05:55 2022] PUT /identity/v3/projects/d34db96dd82a425d90973717021e4c95/users/74ac4fc2e1df423a86a4ee9c1137135a/roles/432bfb33f0224d11acc5bae2c196dc77 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:05:56.622768 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb2f157-15ea-4980-8438-cd3cf1ce3fdb None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:56.622768 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb2f157-15ea-4980-8438-cd3cf1ce3fdb None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:56.622768 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb2f157-15ea-4980-8438-cd3cf1ce3fdb None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:56.622768 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 144/287] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:05:56.628498 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508cabd-86c3-48fa-9d5f-c7082d033c53 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:56.629465 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508cabd-86c3-48fa-9d5f-c7082d033c53 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:56.629943 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508cabd-86c3-48fa-9d5f-c7082d033c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:56.649994 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-a508cabd-86c3-48fa-9d5f-c7082d033c53 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:56.651562 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-a508cabd-86c3-48fa-9d5f-c7082d033c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:56.736180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 144/288] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 06:05:56.759791 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb2f157-15ea-4980-8438-cd3cf1ce3fdb None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:56.774465 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:56.902236 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb2f157-15ea-4980-8438-cd3cf1ce3fdb None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:56.950566 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948bbe1f-c1ba-487f-a9bb-ebea42f4b01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:56.951372 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948bbe1f-c1ba-487f-a9bb-ebea42f4b01e None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:56.951661 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948bbe1f-c1ba-487f-a9bb-ebea42f4b01e None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:56.951939 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948bbe1f-c1ba-487f-a9bb-ebea42f4b01e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:56.968652 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948bbe1f-c1ba-487f-a9bb-ebea42f4b01e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:05:56.976772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-948bbe1f-c1ba-487f-a9bb-ebea42f4b01e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 06:05:56.976772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:05:56.976772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:05:56.976772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:05:56.976772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:05:56.976772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:05:56.976772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:05:56.976772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:05:56.976772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:05:56.976772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:05:56.976772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:05:56.976772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:05:56.976772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:05:56.976772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:05:56.976772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:05:56.976772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:05:56.977649 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:05:56.977649 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:05:56.977649 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:05:56.977649 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:05:56.977649 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:05:56.977649 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:05:56.977649 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:05:56.977649 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:05:56.977649 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:05:56.977649 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:05:56.977649 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:05:56.977649 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:05:56.977649 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:05:56.977649 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:05:56.977649 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:05:56.977649 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:05:56.978425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:05:56.978425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:05:56.978425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:05:56.978425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:05:56.978425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:05:56.978425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:05:56.978425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:05:56.978425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:05:56.978425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:05:56.978425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:05:56.978425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 06:05:56.978425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:05:56.978425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 145/289] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:05:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:05:56.985642 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a508cabd-86c3-48fa-9d5f-c7082d033c53 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:57.055954 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:57.099525 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a508cabd-86c3-48fa-9d5f-c7082d033c53 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:57.137426 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d7eb4-e2c1-4ce0-b449-25a91e7f04f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:57.138104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d7eb4-e2c1-4ce0-b449-25a91e7f04f9 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:57.138394 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d7eb4-e2c1-4ce0-b449-25a91e7f04f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:57.138674 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d7eb4-e2c1-4ce0-b449-25a91e7f04f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:57.139442 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d7eb4-e2c1-4ce0-b449-25a91e7f04f9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:57.139751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d7eb4-e2c1-4ce0-b449-25a91e7f04f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:57.141241 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d7eb4-e2c1-4ce0-b449-25a91e7f04f9 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:57.156364 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 145/290] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 18 06:05:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:57.157700 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948bbe1f-c1ba-487f-a9bb-ebea42f4b01e None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:57.260459 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:57.306580 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948bbe1f-c1ba-487f-a9bb-ebea42f4b01e None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:57.358197 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d07df-ece1-4e54-8b44-b71986cc7b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:57.359138 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d07df-ece1-4e54-8b44-b71986cc7b4d None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:57.359429 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d07df-ece1-4e54-8b44-b71986cc7b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:57.359708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d07df-ece1-4e54-8b44-b71986cc7b4d None admin] PATH_INFO: `/v3/users/70ecfa9b1920435485830e0e7e41c2d8` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:57.360944 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d07df-ece1-4e54-8b44-b71986cc7b4d None admin] RBAC: Authorizing `identity:get_user(user_id=70ecfa9b1920435485830e0e7e41c2d8)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:57.362473 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d07df-ece1-4e54-8b44-b71986cc7b4d None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:57.364489 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 146/291] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:05:57 2022] GET /identity/v3/users/70ecfa9b1920435485830e0e7e41c2d8 => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:57.370129 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d7eb4-e2c1-4ce0-b449-25a91e7f04f9 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:57.482029 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d7eb4-e2c1-4ce0-b449-25a91e7f04f9 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:57.506024 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:57.518547 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead74bf-a75e-4acb-bd06-94503dc748ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:57.519214 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead74bf-a75e-4acb-bd06-94503dc748ac None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:57.519508 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead74bf-a75e-4acb-bd06-94503dc748ac None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:57.519791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead74bf-a75e-4acb-bd06-94503dc748ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:57.594435 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fead74bf-a75e-4acb-bd06-94503dc748ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:57.595942 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fead74bf-a75e-4acb-bd06-94503dc748ac None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:57.634195 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e73b5-37fc-4a98-833b-a7496c801da3 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:57.635179 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e73b5-37fc-4a98-833b-a7496c801da3 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:57.636165 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e73b5-37fc-4a98-833b-a7496c801da3 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:57.638103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 147/292] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:05:57.643604 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aad6e7-5de6-4659-8e35-f6034a87280c None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:57.644197 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aad6e7-5de6-4659-8e35-f6034a87280c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:57.644709 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aad6e7-5de6-4659-8e35-f6034a87280c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:57.686623 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-b1aad6e7-5de6-4659-8e35-f6034a87280c None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:57.688168 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b1aad6e7-5de6-4659-8e35-f6034a87280c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:57.708559 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 146/293] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:05:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 06:05:57.710536 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead74bf-a75e-4acb-bd06-94503dc748ac None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:57.736511 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:57.785689 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 148/294] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:05:57.820631 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead74bf-a75e-4acb-bd06-94503dc748ac None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:57.824964 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aad6e7-5de6-4659-8e35-f6034a87280c None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:57.861008 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ce947a-eae7-40a4-9837-0cc7e0505642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:57.862551 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce947a-eae7-40a4-9837-0cc7e0505642 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:57.862977 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce947a-eae7-40a4-9837-0cc7e0505642 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:57.863399 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce947a-eae7-40a4-9837-0cc7e0505642 None admin] PATH_INFO: `/v3/projects/d34db96dd82a425d90973717021e4c95` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:57.864941 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ce947a-eae7-40a4-9837-0cc7e0505642 None admin] RBAC: Authorizing `identity:get_project(project_id=d34db96dd82a425d90973717021e4c95)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:57.866592 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ce947a-eae7-40a4-9837-0cc7e0505642 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:57.872533 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 147/295] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:05:57 2022] GET /identity/v3/projects/d34db96dd82a425d90973717021e4c95 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:57.874674 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37074ae6-c52e-4905-bceb-0a573bf39a52 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:57.875136 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37074ae6-c52e-4905-bceb-0a573bf39a52 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:57.875572 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37074ae6-c52e-4905-bceb-0a573bf39a52 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:57.877098 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 148/296] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:05:57.881824 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37074ae6-c52e-4905-bceb-0a573bf39a52 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:57.935906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:57.992235 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aad6e7-5de6-4659-8e35-f6034a87280c None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:58.001313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:58.015403 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37074ae6-c52e-4905-bceb-0a573bf39a52 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:58.050831 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9ccfc7-d4ba-4f60-97eb-fd53b338c8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:58.051987 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ccfc7-d4ba-4f60-97eb-fd53b338c8ab None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:58.052529 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ccfc7-d4ba-4f60-97eb-fd53b338c8ab None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:58.052988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ccfc7-d4ba-4f60-97eb-fd53b338c8ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:58.054420 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9ccfc7-d4ba-4f60-97eb-fd53b338c8ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:58.056813 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9ccfc7-d4ba-4f60-97eb-fd53b338c8ab None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:58.060753 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 149/297] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:05:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:05:58.061470 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ac926-cb64-4ebf-a91f-bd1d86e4a6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:58.062504 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ac926-cb64-4ebf-a91f-bd1d86e4a6c5 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:58.062829 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35ace3-5fae-4ad5-9209-d1831658c38f None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:58.062949 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ac926-cb64-4ebf-a91f-bd1d86e4a6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:58.063285 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ac926-cb64-4ebf-a91f-bd1d86e4a6c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:58.063587 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35ace3-5fae-4ad5-9209-d1831658c38f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:58.064031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35ace3-5fae-4ad5-9209-d1831658c38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:58.064362 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55ac926-cb64-4ebf-a91f-bd1d86e4a6c5 None admin] RBAC: Adding query filter params (role.id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20, scope.project.id=d34db96dd82a425d90973717021e4c95, user.id=70ecfa9b1920435485830e0e7e41c2d8) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:58.064874 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55ac926-cb64-4ebf-a91f-bd1d86e4a6c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:58.066891 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55ac926-cb64-4ebf-a91f-bd1d86e4a6c5 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:58.084021 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 149/298] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:05:57 2022] GET /identity/v3/role_assignments?user.id=70ecfa9b1920435485830e0e7e41c2d8&scope.project.id=d34db96dd82a425d90973717021e4c95&role.id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:58.085817 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ac926-cb64-4ebf-a91f-bd1d86e4a6c5 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:58.095280 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-2f35ace3-5fae-4ad5-9209-d1831658c38f None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:58.097686 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-2f35ace3-5fae-4ad5-9209-d1831658c38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:58.175707 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 150/299] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:05:58.177477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adae985-5350-4665-bdd8-80fa38f4ebd7 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:58.177913 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adae985-5350-4665-bdd8-80fa38f4ebd7 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:58.178336 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adae985-5350-4665-bdd8-80fa38f4ebd7 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:58.180224 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 151/300] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:05:58.190016 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce8199-3ca4-4147-aaa7-56820362feca None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:58.190540 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce8199-3ca4-4147-aaa7-56820362feca None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:58.190962 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce8199-3ca4-4147-aaa7-56820362feca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:58.216341 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ac926-cb64-4ebf-a91f-bd1d86e4a6c5 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:58.225625 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-33ce8199-3ca4-4147-aaa7-56820362feca None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:58.227191 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-33ce8199-3ca4-4147-aaa7-56820362feca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:58.257993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf3c9a1-23a1-4c94-b7e2-f84c07926349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:58.258756 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3c9a1-23a1-4c94-b7e2-f84c07926349 None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:58.259148 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3c9a1-23a1-4c94-b7e2-f84c07926349 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:58.259534 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3c9a1-23a1-4c94-b7e2-f84c07926349 None admin] PATH_INFO: `/v3/groups` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:58.260398 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf3c9a1-23a1-4c94-b7e2-f84c07926349 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:58.262949 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf3c9a1-23a1-4c94-b7e2-f84c07926349 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:58.303335 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:58.311821 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 152/301] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:05:58.318098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ce8199-3ca4-4147-aaa7-56820362feca None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:58.347301 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 150/302] 10.222.0.40 () {60 vars in 1134 bytes} [Mon Jul 18 06:05:58 2022] POST /identity/v3/groups => generated 229 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:05:58.348154 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf3c9a1-23a1-4c94-b7e2-f84c07926349 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:58.371745 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:58.485759 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ce8199-3ca4-4147-aaa7-56820362feca None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:58.524836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf3c9a1-23a1-4c94-b7e2-f84c07926349 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:58.571958 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e8bd4b-178d-455f-83bb-683e2f355a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:58.572881 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e8bd4b-178d-455f-83bb-683e2f355a23 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:58.573691 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e8bd4b-178d-455f-83bb-683e2f355a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:58.574093 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e8bd4b-178d-455f-83bb-683e2f355a23 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:58.590998 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e8bd4b-178d-455f-83bb-683e2f355a23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:05:58.593743 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415cb026-f57b-4d7e-84cc-27da951673e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:58.593959 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cb026-f57b-4d7e-84cc-27da951673e9 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:58.594203 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cb026-f57b-4d7e-84cc-27da951673e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:58.594494 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cb026-f57b-4d7e-84cc-27da951673e9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:58.600103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-c4e8bd4b-178d-455f-83bb-683e2f355a23 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 06:05:58.600103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:05:58.600103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:05:58.600103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:05:58.600103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:05:58.600103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:05:58.600103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:05:58.600103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:05:58.600103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:05:58.600103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:05:58.600103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:05:58.600103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:05:58.600103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:05:58.600103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:05:58.600103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:05:58.600103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:05:58.601288 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:05:58.601288 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:05:58.601288 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:05:58.601288 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:05:58.601288 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:05:58.601288 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:05:58.601288 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:05:58.601288 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:05:58.601288 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:05:58.601288 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:05:58.601288 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:05:58.601288 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:05:58.601288 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:05:58.601288 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:05:58.601288 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:05:58.601288 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:05:58.602476 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:05:58.602476 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:05:58.602476 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:05:58.602476 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:05:58.602476 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:05:58.602476 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:05:58.602476 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:05:58.602476 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:05:58.602476 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:05:58.602476 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:05:58.602476 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 06:05:58.602476 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:05:58.602476 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 153/303] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:05:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:05:58.607709 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e8bd4b-178d-455f-83bb-683e2f355a23 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:58.611190 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415cb026-f57b-4d7e-84cc-27da951673e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:05:58.623454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-415cb026-f57b-4d7e-84cc-27da951673e9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 06:05:58.623454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:05:58.623454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:05:58.623454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:05:58.623454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:05:58.623454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:05:58.623454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:05:58.623454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:05:58.623454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:05:58.623454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:05:58.623454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:05:58.623454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:05:58.623454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:05:58.623454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:05:58.623454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:05:58.623454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:05:58.624801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:05:58.624801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:05:58.624801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:05:58.624801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:05:58.624801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:05:58.624801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:05:58.624801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:05:58.624801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:05:58.624801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:05:58.624801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:05:58.624801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:05:58.624801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:05:58.624801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:05:58.624801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:05:58.624801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:05:58.624801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:05:58.625761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:05:58.625761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:05:58.625761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:05:58.625761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:05:58.625761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:05:58.625761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:05:58.625761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:05:58.625761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:05:58.625761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:05:58.625761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:05:58.625761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 06:05:58.625761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:05:58.625761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 151/304] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:05:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:05:58.631793 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415cb026-f57b-4d7e-84cc-27da951673e9 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:58.685888 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:58.745276 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:58.745857 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e8bd4b-178d-455f-83bb-683e2f355a23 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:58.781018 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415cb026-f57b-4d7e-84cc-27da951673e9 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:58.790772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a2e170-23c0-43a7-ad26-318a07f001c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:58.791456 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a2e170-23c0-43a7-ad26-318a07f001c1 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:58.791964 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a2e170-23c0-43a7-ad26-318a07f001c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:58.792743 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a2e170-23c0-43a7-ad26-318a07f001c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:58.793440 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a2e170-23c0-43a7-ad26-318a07f001c1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:58.793744 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a2e170-23c0-43a7-ad26-318a07f001c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:58.795161 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a2e170-23c0-43a7-ad26-318a07f001c1 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:58.806326 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 154/305] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 18 06:05:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:05:58.815559 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a2e170-23c0-43a7-ad26-318a07f001c1 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:58.857000 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da973d5-6c0f-4f99-bf3c-3d3587153147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:58.857697 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da973d5-6c0f-4f99-bf3c-3d3587153147 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:58.860449 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da973d5-6c0f-4f99-bf3c-3d3587153147 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:58.860449 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da973d5-6c0f-4f99-bf3c-3d3587153147 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:58.860449 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da973d5-6c0f-4f99-bf3c-3d3587153147 None admin] RBAC: Adding query filter params (name=member) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:58.860449 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da973d5-6c0f-4f99-bf3c-3d3587153147 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:58.860903 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da973d5-6c0f-4f99-bf3c-3d3587153147 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:58.882219 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 152/306] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 18 06:05:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:58.884893 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da973d5-6c0f-4f99-bf3c-3d3587153147 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:58.998879 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da973d5-6c0f-4f99-bf3c-3d3587153147 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:59.038383 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fabef0-5d21-4db0-8b99-00b45a118ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:59.044442 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fabef0-5d21-4db0-8b99-00b45a118ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:59.044442 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fabef0-5d21-4db0-8b99-00b45a118ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:59.044442 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fabef0-5d21-4db0-8b99-00b45a118ba7 None admin] PATH_INFO: `/v3/users/6228fb05d5d74f6babe66d0649479859` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:59.044442 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fabef0-5d21-4db0-8b99-00b45a118ba7 None admin] RBAC: Authorizing `identity:get_user(user_id=6228fb05d5d74f6babe66d0649479859)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:59.044442 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fabef0-5d21-4db0-8b99-00b45a118ba7 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:59.045040 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 153/307] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:05:58 2022] GET /identity/v3/users/6228fb05d5d74f6babe66d0649479859 => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:05:59.050685 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fabef0-5d21-4db0-8b99-00b45a118ba7 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:59.069972 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:59.145652 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a2e170-23c0-43a7-ad26-318a07f001c1 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:59.168033 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:59.183474 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fabef0-5d21-4db0-8b99-00b45a118ba7 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:59.187813 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab67be1-8e44-4296-9812-bf17396a9372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:59.188814 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab67be1-8e44-4296-9812-bf17396a9372 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:59.189103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab67be1-8e44-4296-9812-bf17396a9372 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:59.189385 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab67be1-8e44-4296-9812-bf17396a9372 None admin] PATH_INFO: `/v3/users/f4524ff1ba3344b9b506631806564d55` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:59.206501 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab67be1-8e44-4296-9812-bf17396a9372 None admin] RBAC: Authorizing `identity:get_user(user_id=f4524ff1ba3344b9b506631806564d55)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:59.208360 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab67be1-8e44-4296-9812-bf17396a9372 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:59.212583 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 155/308] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:05:58 2022] GET /identity/v3/users/f4524ff1ba3344b9b506631806564d55 => generated 293 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:59.214919 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a65e8-3540-4b5b-bee2-65d5f519fa1e None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:59.215222 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a65e8-3540-4b5b-bee2-65d5f519fa1e None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:59.215479 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a65e8-3540-4b5b-bee2-65d5f519fa1e None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:59.217040 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 156/309] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:05:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:05:59.217931 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226a65e8-3540-4b5b-bee2-65d5f519fa1e None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:59.233495 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501132d-9420-40d9-b174-11f65ec8a84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:59.234706 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501132d-9420-40d9-b174-11f65ec8a84b None admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:59.234991 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501132d-9420-40d9-b174-11f65ec8a84b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:59.235277 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501132d-9420-40d9-b174-11f65ec8a84b None admin] PATH_INFO: `/v3/system/users/6228fb05d5d74f6babe66d0649479859/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:59.236885 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d501132d-9420-40d9-b174-11f65ec8a84b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6228fb05d5d74f6babe66d0649479859, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:59.238395 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d501132d-9420-40d9-b174-11f65ec8a84b None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:59.329433 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226a65e8-3540-4b5b-bee2-65d5f519fa1e None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:59.347981 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 154/310] 10.222.0.40 () {58 vars in 1334 bytes} [Mon Jul 18 06:05:59 2022] PUT /identity/v3/system/users/6228fb05d5d74f6babe66d0649479859/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:05:59.349403 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8d912-fc2d-4832-bb39-bbda284e9f53 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:59.349690 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8d912-fc2d-4832-bb39-bbda284e9f53 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:59.349968 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8d912-fc2d-4832-bb39-bbda284e9f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:59.375736 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9b18b-d239-4fe3-8e90-57504626cc63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:59.376426 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9b18b-d239-4fe3-8e90-57504626cc63 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:59.376727 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9b18b-d239-4fe3-8e90-57504626cc63 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:59.377010 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9b18b-d239-4fe3-8e90-57504626cc63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:59.377576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-ccf8d912-fc2d-4832-bb39-bbda284e9f53 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:05:59.377887 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e9b18b-d239-4fe3-8e90-57504626cc63 None admin] RBAC: Adding query filter params (role.id=432bfb33f0224d11acc5bae2c196dc77, user.id=f4524ff1ba3344b9b506631806564d55, scope.system=all) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:59.378213 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e9b18b-d239-4fe3-8e90-57504626cc63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:59.379673 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ccf8d912-fc2d-4832-bb39-bbda284e9f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:05:59.379803 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e9b18b-d239-4fe3-8e90-57504626cc63 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:59.387323 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 157/311] 10.222.0.40 () {56 vars in 1308 bytes} [Mon Jul 18 06:05:59 2022] GET /identity/v3/role_assignments?user.id=f4524ff1ba3344b9b506631806564d55&scope.system=all&role.id=432bfb33f0224d11acc5bae2c196dc77 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:59.464043 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:59.471874 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 155/312] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:05:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:05:59.495859 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9b18b-d239-4fe3-8e90-57504626cc63 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:59.551759 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:59.630773 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9b18b-d239-4fe3-8e90-57504626cc63 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:59.670558 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25bce93-fe79-462d-a581-a945f948d154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:59.671393 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25bce93-fe79-462d-a581-a945f948d154 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:59.671677 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25bce93-fe79-462d-a581-a945f948d154 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:59.671953 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25bce93-fe79-462d-a581-a945f948d154 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:59.688870 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25bce93-fe79-462d-a581-a945f948d154 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:05:59.696689 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-d25bce93-fe79-462d-a581-a945f948d154 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 06:05:59.696689 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:05:59.696689 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:05:59.696689 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:05:59.696689 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:05:59.696689 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:05:59.696689 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:05:59.696689 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:05:59.696689 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:05:59.696689 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:05:59.696689 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:05:59.696689 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:05:59.696689 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:05:59.696689 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:05:59.696689 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:05:59.696689 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:05:59.697664 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:05:59.697664 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:05:59.697664 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:05:59.697664 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:05:59.697664 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:05:59.697664 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:05:59.697664 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:05:59.697664 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:05:59.697664 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:05:59.697664 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:05:59.697664 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:05:59.697664 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:05:59.697664 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:05:59.697664 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:05:59.697664 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:05:59.697664 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:05:59.699674 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:05:59.699674 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:05:59.699674 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:05:59.699674 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:05:59.699674 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:05:59.699674 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:05:59.699674 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:05:59.699674 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:05:59.699674 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:05:59.699674 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:05:59.699674 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 06:05:59.699674 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:05:59.699674 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 158/313] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:05:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:05:59.704713 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf8d912-fc2d-4832-bb39-bbda284e9f53 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:59.824016 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf8d912-fc2d-4832-bb39-bbda284e9f53 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:05:59.862143 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c929988-2741-4e9f-8b2e-2718c7b9da0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:05:59.862833 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c929988-2741-4e9f-8b2e-2718c7b9da0d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:05:59.863122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c929988-2741-4e9f-8b2e-2718c7b9da0d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:05:59.863405 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c929988-2741-4e9f-8b2e-2718c7b9da0d None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:05:59.864172 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c929988-2741-4e9f-8b2e-2718c7b9da0d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:05:59.864511 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c929988-2741-4e9f-8b2e-2718c7b9da0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:05:59.865970 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c929988-2741-4e9f-8b2e-2718c7b9da0d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:05:59.876702 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 156/314] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 18 06:05:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:05:59.881993 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25bce93-fe79-462d-a581-a945f948d154 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:05:59.994075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:05:59.998752 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25bce93-fe79-462d-a581-a945f948d154 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:00.034945 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f10b6-c45e-4901-90c1-8d14baf818e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:00.035829 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f10b6-c45e-4901-90c1-8d14baf818e9 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:00.036120 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f10b6-c45e-4901-90c1-8d14baf818e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:00.036404 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f10b6-c45e-4901-90c1-8d14baf818e9 None admin] PATH_INFO: `/v3/users/74ac4fc2e1df423a86a4ee9c1137135a` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:00.037987 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55f10b6-c45e-4901-90c1-8d14baf818e9 None admin] RBAC: Authorizing `identity:get_user(user_id=74ac4fc2e1df423a86a4ee9c1137135a)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:00.039620 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55f10b6-c45e-4901-90c1-8d14baf818e9 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:00.042435 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 159/315] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:05:59 2022] GET /identity/v3/users/74ac4fc2e1df423a86a4ee9c1137135a => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:00.048575 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c929988-2741-4e9f-8b2e-2718c7b9da0d None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:00.114232 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:00.164235 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c929988-2741-4e9f-8b2e-2718c7b9da0d None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:00.202292 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec132d4-2fa0-4ef8-87b1-07a96554fe81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:00.202960 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec132d4-2fa0-4ef8-87b1-07a96554fe81 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:00.203251 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec132d4-2fa0-4ef8-87b1-07a96554fe81 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:00.203537 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec132d4-2fa0-4ef8-87b1-07a96554fe81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:00.279088 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec132d4-2fa0-4ef8-87b1-07a96554fe81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:00.280611 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec132d4-2fa0-4ef8-87b1-07a96554fe81 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:00.344341 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:00.390656 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 157/316] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:06:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 06:06:00.400003 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f10b6-c45e-4901-90c1-8d14baf818e9 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:00.508879 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f10b6-c45e-4901-90c1-8d14baf818e9 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:00.545035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ae11c-a5cc-4519-9626-fb4f38e13b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:00.545909 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ae11c-a5cc-4519-9626-fb4f38e13b9e None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:00.546201 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ae11c-a5cc-4519-9626-fb4f38e13b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:00.546483 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ae11c-a5cc-4519-9626-fb4f38e13b9e None admin] PATH_INFO: `/v3/projects/d34db96dd82a425d90973717021e4c95` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:00.547688 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ae11c-a5cc-4519-9626-fb4f38e13b9e None admin] RBAC: Authorizing `identity:get_project(project_id=d34db96dd82a425d90973717021e4c95)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:00.549168 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ae11c-a5cc-4519-9626-fb4f38e13b9e None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:00.551249 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 160/317] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:06:00 2022] GET /identity/v3/projects/d34db96dd82a425d90973717021e4c95 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:00.556767 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec132d4-2fa0-4ef8-87b1-07a96554fe81 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:00.602294 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ff841-863d-4b51-a27d-55aca1fb561c None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:00.602621 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ff841-863d-4b51-a27d-55aca1fb561c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:00.602849 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ff841-863d-4b51-a27d-55aca1fb561c None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:00.604375 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 161/318] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:06:00.610010 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b84f8-1cb9-4c93-85ba-fe0fb5064a20 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:00.610287 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b84f8-1cb9-4c93-85ba-fe0fb5064a20 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:00.610564 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b84f8-1cb9-4c93-85ba-fe0fb5064a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:00.629950 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-fb6b84f8-1cb9-4c93-85ba-fe0fb5064a20 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:00.632081 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-fb6b84f8-1cb9-4c93-85ba-fe0fb5064a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:00.677509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:00.697848 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec132d4-2fa0-4ef8-87b1-07a96554fe81 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:00.734733 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 162/319] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:06:00.749866 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1ce06-db6c-4926-acc6-cdffea810096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:00.750767 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1ce06-db6c-4926-acc6-cdffea810096 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:00.751194 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1ce06-db6c-4926-acc6-cdffea810096 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:00.751615 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1ce06-db6c-4926-acc6-cdffea810096 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:00.752640 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf1ce06-db6c-4926-acc6-cdffea810096 None admin] RBAC: Adding query filter params (role.id=432bfb33f0224d11acc5bae2c196dc77, scope.project.id=d34db96dd82a425d90973717021e4c95, user.id=74ac4fc2e1df423a86a4ee9c1137135a) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:00.753103 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf1ce06-db6c-4926-acc6-cdffea810096 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:00.754692 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf1ce06-db6c-4926-acc6-cdffea810096 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:00.758102 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b84f8-1cb9-4c93-85ba-fe0fb5064a20 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:00.765490 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 158/320] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:06:00 2022] GET /identity/v3/role_assignments?user.id=74ac4fc2e1df423a86a4ee9c1137135a&scope.project.id=d34db96dd82a425d90973717021e4c95&role.id=432bfb33f0224d11acc5bae2c196dc77 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:00.883174 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b84f8-1cb9-4c93-85ba-fe0fb5064a20 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:00.923805 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e973349c-67d1-4521-a51b-5459a224354c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:00.924628 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973349c-67d1-4521-a51b-5459a224354c None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:00.924956 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973349c-67d1-4521-a51b-5459a224354c None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:00.925231 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973349c-67d1-4521-a51b-5459a224354c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:00.926400 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e973349c-67d1-4521-a51b-5459a224354c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:00.927837 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e973349c-67d1-4521-a51b-5459a224354c None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:00.929644 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 163/321] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:06:00.935091 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1ce06-db6c-4926-acc6-cdffea810096 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:00.963521 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:01.047298 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1ce06-db6c-4926-acc6-cdffea810096 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:01.082026 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c18af-676c-4af8-961e-c32db3d370da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:01.082699 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c18af-676c-4af8-961e-c32db3d370da None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:01.082983 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c18af-676c-4af8-961e-c32db3d370da None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:01.083259 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c18af-676c-4af8-961e-c32db3d370da None admin] PATH_INFO: `/v3/groups` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:01.083961 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147c18af-676c-4af8-961e-c32db3d370da None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:01.085422 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147c18af-676c-4af8-961e-c32db3d370da None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:01.183477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 159/322] 10.222.0.40 () {60 vars in 1134 bytes} [Mon Jul 18 06:06:00 2022] POST /identity/v3/groups => generated 226 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:06:01.584029 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f91630c-1f8f-42d5-887b-08442108780e None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:01.584759 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f91630c-1f8f-42d5-887b-08442108780e None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:01.585258 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f91630c-1f8f-42d5-887b-08442108780e None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:01.587161 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 164/323] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:01.593625 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496fbc5e-d0e9-4ccb-aaf5-c41c978a84d1 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:01.593923 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496fbc5e-d0e9-4ccb-aaf5-c41c978a84d1 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:01.594208 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496fbc5e-d0e9-4ccb-aaf5-c41c978a84d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:01.612188 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-496fbc5e-d0e9-4ccb-aaf5-c41c978a84d1 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:01.613439 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-496fbc5e-d0e9-4ccb-aaf5-c41c978a84d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:01.645889 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb5e22-3836-4985-aaba-a29fd7baf09c None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:01.646539 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb5e22-3836-4985-aaba-a29fd7baf09c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:01.646922 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb5e22-3836-4985-aaba-a29fd7baf09c None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:01.648667 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 165/324] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:01.656294 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a378c86-60ea-4cd5-b9ab-2eb39c47ab38 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:01.656549 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a378c86-60ea-4cd5-b9ab-2eb39c47ab38 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:01.656840 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a378c86-60ea-4cd5-b9ab-2eb39c47ab38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:01.687135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-4a378c86-60ea-4cd5-b9ab-2eb39c47ab38 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:01.688511 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4a378c86-60ea-4cd5-b9ab-2eb39c47ab38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:01.705265 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 160/325] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:06:01.724571 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:01.733589 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496fbc5e-d0e9-4ccb-aaf5-c41c978a84d1 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:01.768851 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:01.777942 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 166/326] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 06:06:01.802343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a378c86-60ea-4cd5-b9ab-2eb39c47ab38 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:01.862976 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496fbc5e-d0e9-4ccb-aaf5-c41c978a84d1 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:01.906041 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ddbea1-f43f-479a-89b5-0cea7fd59ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:01.907063 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ddbea1-f43f-479a-89b5-0cea7fd59ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:01.907358 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ddbea1-f43f-479a-89b5-0cea7fd59ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:01.907640 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ddbea1-f43f-479a-89b5-0cea7fd59ef7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:01.923188 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ddbea1-f43f-479a-89b5-0cea7fd59ef7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:01.936185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-68ddbea1-f43f-479a-89b5-0cea7fd59ef7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 06:06:01.936185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:01.936185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:01.936185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:01.936185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:01.936185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:01.936185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:01.936185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:01.936185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:01.936185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:01.936185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:01.936185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:01.936185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:01.936185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:01.936185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:01.936185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:01.937601 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:01.937601 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:01.937601 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:01.937601 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:01.937601 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:01.937601 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:01.937601 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:01.937601 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:01.937601 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:01.937601 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:01.937601 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:01.937601 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:01.937601 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:01.937601 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:01.937601 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:01.937601 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:01.938629 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:01.938629 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:01.938629 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:01.938629 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:01.938629 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:01.938629 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:01.938629 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:01.938629 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:01.938629 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:01.938629 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:01.938629 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 06:06:01.938629 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:01.938629 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 161/327] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:06:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:06:01.939848 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a378c86-60ea-4cd5-b9ab-2eb39c47ab38 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:01.946126 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ddbea1-f43f-479a-89b5-0cea7fd59ef7 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:01.990589 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8b571-c646-494f-9397-059b46df0909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:01.991982 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8b571-c646-494f-9397-059b46df0909 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:01.992272 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8b571-c646-494f-9397-059b46df0909 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:01.992583 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8b571-c646-494f-9397-059b46df0909 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:02.007742 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e8b571-c646-494f-9397-059b46df0909 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:02.023677 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-74e8b571-c646-494f-9397-059b46df0909 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 06:06:02.023677 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:02.023677 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:02.023677 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:02.023677 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:02.023677 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:02.023677 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:02.023677 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:02.023677 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:02.023677 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:02.023677 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:02.023677 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:02.023677 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:02.023677 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:02.023677 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:02.023677 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:02.024706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:02.024706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:02.024706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:02.024706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:02.024706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:02.024706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:02.024706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:02.024706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:02.024706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:02.024706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:02.024706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:02.024706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:02.024706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:02.024706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:02.024706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:02.024706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:02.025507 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:02.025507 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:02.025507 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:02.025507 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:02.025507 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:02.025507 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:02.025507 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:02.025507 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:02.025507 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:02.025507 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:02.025507 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 06:06:02.025507 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:02.025507 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 167/328] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:06:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:02.034322 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8b571-c646-494f-9397-059b46df0909 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:02.058063 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:02.069214 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:02.085530 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ddbea1-f43f-479a-89b5-0cea7fd59ef7 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:02.131521 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6adfe8f-f84c-4e52-80c9-393559045d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:02.132596 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6adfe8f-f84c-4e52-80c9-393559045d22 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:02.133205 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6adfe8f-f84c-4e52-80c9-393559045d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:02.133632 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6adfe8f-f84c-4e52-80c9-393559045d22 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:02.134699 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6adfe8f-f84c-4e52-80c9-393559045d22 None admin] RBAC: Adding query filter params (name=member) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:02.135008 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6adfe8f-f84c-4e52-80c9-393559045d22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:02.136519 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6adfe8f-f84c-4e52-80c9-393559045d22 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:02.146159 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 162/329] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 18 06:06:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:02.154254 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6adfe8f-f84c-4e52-80c9-393559045d22 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:02.175367 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8b571-c646-494f-9397-059b46df0909 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:02.250335 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2c7e61-ed1c-4618-9832-1252508720ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:02.251019 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c7e61-ed1c-4618-9832-1252508720ad None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:02.251307 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c7e61-ed1c-4618-9832-1252508720ad None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:02.251588 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c7e61-ed1c-4618-9832-1252508720ad None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:02.252365 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2c7e61-ed1c-4618-9832-1252508720ad None admin] RBAC: Adding query filter params (name=reader) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:02.252771 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2c7e61-ed1c-4618-9832-1252508720ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:02.254148 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2c7e61-ed1c-4618-9832-1252508720ad None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:02.268856 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 168/330] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 18 06:06:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:06:02.274645 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2c7e61-ed1c-4618-9832-1252508720ad None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:02.351239 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6adfe8f-f84c-4e52-80c9-393559045d22 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:02.422831 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d601af-2a24-4cba-9fe3-2c6bd21dfcb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:02.423707 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d601af-2a24-4cba-9fe3-2c6bd21dfcb2 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:02.423994 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d601af-2a24-4cba-9fe3-2c6bd21dfcb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:02.424269 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d601af-2a24-4cba-9fe3-2c6bd21dfcb2 None admin] PATH_INFO: `/v3/users/6228fb05d5d74f6babe66d0649479859` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:02.426719 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d601af-2a24-4cba-9fe3-2c6bd21dfcb2 None admin] RBAC: Authorizing `identity:get_user(user_id=6228fb05d5d74f6babe66d0649479859)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:02.428221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d601af-2a24-4cba-9fe3-2c6bd21dfcb2 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:02.436848 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 163/331] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:06:02 2022] GET /identity/v3/users/6228fb05d5d74f6babe66d0649479859 => generated 293 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:06:02.443359 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d601af-2a24-4cba-9fe3-2c6bd21dfcb2 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:02.449224 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:02.449719 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:02.483321 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2c7e61-ed1c-4618-9832-1252508720ad None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:02.562729 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c24f08-6b87-4f21-ab05-82aa7e7a9bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:02.563647 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c24f08-6b87-4f21-ab05-82aa7e7a9bbc None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:02.563934 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c24f08-6b87-4f21-ab05-82aa7e7a9bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:02.564282 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c24f08-6b87-4f21-ab05-82aa7e7a9bbc None admin] PATH_INFO: `/v3/users/f4524ff1ba3344b9b506631806564d55` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:02.565833 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c24f08-6b87-4f21-ab05-82aa7e7a9bbc None admin] RBAC: Authorizing `identity:get_user(user_id=f4524ff1ba3344b9b506631806564d55)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:02.567570 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c24f08-6b87-4f21-ab05-82aa7e7a9bbc None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:02.569531 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 169/332] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:06:02 2022] GET /identity/v3/users/f4524ff1ba3344b9b506631806564d55 => generated 293 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:06:02.578383 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c24f08-6b87-4f21-ab05-82aa7e7a9bbc None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:02.666759 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d601af-2a24-4cba-9fe3-2c6bd21dfcb2 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:02.730921 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc72d8-4b03-4158-a636-3545660e76c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:02.731611 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc72d8-4b03-4158-a636-3545660e76c8 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:02.731903 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc72d8-4b03-4158-a636-3545660e76c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:02.732184 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc72d8-4b03-4158-a636-3545660e76c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:02.733071 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fc72d8-4b03-4158-a636-3545660e76c8 None admin] RBAC: Adding query filter params (role.id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20, user.id=6228fb05d5d74f6babe66d0649479859, scope.system=all) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:02.733941 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fc72d8-4b03-4158-a636-3545660e76c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:02.735593 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fc72d8-4b03-4158-a636-3545660e76c8 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:02.761017 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 164/333] 10.222.0.40 () {56 vars in 1308 bytes} [Mon Jul 18 06:06:02 2022] GET /identity/v3/role_assignments?user.id=6228fb05d5d74f6babe66d0649479859&scope.system=all&role.id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 510 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:06:02.810720 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c24f08-6b87-4f21-ab05-82aa7e7a9bbc None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:02.865181 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc10868-a56a-4ae3-8813-fc5efa8bc0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:02.866079 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc10868-a56a-4ae3-8813-fc5efa8bc0d9 None admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:02.866365 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc10868-a56a-4ae3-8813-fc5efa8bc0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:02.866646 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc10868-a56a-4ae3-8813-fc5efa8bc0d9 None admin] PATH_INFO: `/v3/system/users/f4524ff1ba3344b9b506631806564d55/roles/432bfb33f0224d11acc5bae2c196dc77` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:02.868286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc10868-a56a-4ae3-8813-fc5efa8bc0d9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f4524ff1ba3344b9b506631806564d55, role_id=432bfb33f0224d11acc5bae2c196dc77)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:02.869725 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc10868-a56a-4ae3-8813-fc5efa8bc0d9 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:02.948727 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 170/334] 10.222.0.40 () {58 vars in 1334 bytes} [Mon Jul 18 06:06:02 2022] PUT /identity/v3/system/users/f4524ff1ba3344b9b506631806564d55/roles/432bfb33f0224d11acc5bae2c196dc77 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:06:03.979083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df156bd9-7af4-4514-8f10-20c86bf456ff None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:03.980575 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df156bd9-7af4-4514-8f10-20c86bf456ff None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:03.981009 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df156bd9-7af4-4514-8f10-20c86bf456ff None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:03.982911 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 165/335] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:03.990010 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ad34e-7745-40a5-b956-9cb2ff2b17d1 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:03.990507 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ad34e-7745-40a5-b956-9cb2ff2b17d1 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:03.990598 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ad34e-7745-40a5-b956-9cb2ff2b17d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:04.013536 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-754ad34e-7745-40a5-b956-9cb2ff2b17d1 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:04.014999 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-754ad34e-7745-40a5-b956-9cb2ff2b17d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:04.030713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:04.093932 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 171/336] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:06:04.108466 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f3c26-fcfa-409f-a278-6f2e16940c2c None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:04.108466 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f3c26-fcfa-409f-a278-6f2e16940c2c None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:04.108466 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f3c26-fcfa-409f-a278-6f2e16940c2c None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:04.109320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 166/337] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:04.114366 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa530d-b3cb-45e9-a180-e93ff15b2dad None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:04.114654 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa530d-b3cb-45e9-a180-e93ff15b2dad None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:04.114935 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa530d-b3cb-45e9-a180-e93ff15b2dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:04.124225 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f3c26-fcfa-409f-a278-6f2e16940c2c None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:04.141907 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-ebaa530d-b3cb-45e9-a180-e93ff15b2dad None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:04.143115 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ebaa530d-b3cb-45e9-a180-e93ff15b2dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:04.219987 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:04.295034 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f3c26-fcfa-409f-a278-6f2e16940c2c None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:04.302807 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 172/338] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:06:04.339700 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa530d-b3cb-45e9-a180-e93ff15b2dad None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:04.360558 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ddaa4-6dcb-4f0a-bc3d-6917fbc168dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:04.361449 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ddaa4-6dcb-4f0a-bc3d-6917fbc168dc None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:04.361742 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ddaa4-6dcb-4f0a-bc3d-6917fbc168dc None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:04.362032 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ddaa4-6dcb-4f0a-bc3d-6917fbc168dc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:04.381309 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3ddaa4-6dcb-4f0a-bc3d-6917fbc168dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:04.391477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-6e3ddaa4-6dcb-4f0a-bc3d-6917fbc168dc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 06:06:04.391477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:04.391477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:04.391477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:04.391477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:04.391477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:04.391477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:04.391477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:04.391477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:04.391477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:04.391477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:04.391477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:04.391477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:04.391477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:04.391477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:04.391477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:04.392514 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:04.392514 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:04.392514 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:04.392514 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:04.392514 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:04.392514 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:04.392514 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:04.392514 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:04.392514 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:04.392514 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:04.392514 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:04.392514 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:04.392514 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:04.392514 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:04.392514 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:04.392514 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:04.393313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:04.393313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:04.393313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:04.393313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:04.393313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:04.393313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:04.393313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:04.393313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:04.393313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:04.393313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:04.393313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 06:06:04.393313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:04.395582 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 167/339] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:06:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:04.400908 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ddaa4-6dcb-4f0a-bc3d-6917fbc168dc None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:04.508348 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:04.536435 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa530d-b3cb-45e9-a180-e93ff15b2dad None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:04.580857 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12838b6-49c2-4b2a-b771-c04423930738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:04.581700 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12838b6-49c2-4b2a-b771-c04423930738 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:04.581982 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12838b6-49c2-4b2a-b771-c04423930738 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:04.582528 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12838b6-49c2-4b2a-b771-c04423930738 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:04.603971 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12838b6-49c2-4b2a-b771-c04423930738 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:04.616189 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ddaa4-6dcb-4f0a-bc3d-6917fbc168dc None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:04.629327 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-f12838b6-49c2-4b2a-b771-c04423930738 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 06:06:04.629327 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:04.629327 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:04.629327 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:04.629327 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:04.629327 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:04.629327 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:04.629327 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:04.629327 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:04.629327 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:04.629327 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:04.629327 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:04.629327 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:04.629327 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:04.629327 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:04.629327 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:04.630276 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:04.630276 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:04.630276 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:04.630276 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:04.630276 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:04.630276 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:04.630276 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:04.630276 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:04.630276 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:04.630276 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:04.630276 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:04.630276 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:04.630276 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:04.630276 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:04.630276 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:04.630276 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:04.631076 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:04.631076 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:04.631076 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:04.631076 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:04.631076 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:04.631076 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:04.631076 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:04.631076 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:04.631076 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:04.631076 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:04.631076 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 06:06:04.631076 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:04.631076 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 173/340] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 18 06:06:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:04.632254 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800f46c-6826-44ad-8cbb-168b7ce6b0c7 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:04.632254 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800f46c-6826-44ad-8cbb-168b7ce6b0c7 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:04.632551 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800f46c-6826-44ad-8cbb-168b7ce6b0c7 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:04.640427 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 174/341] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:04 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:04.641353 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e800f46c-6826-44ad-8cbb-168b7ce6b0c7 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:04.677430 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:04.702633 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88abb46e-9cf5-46cf-a302-a2503919be55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:04.703320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88abb46e-9cf5-46cf-a302-a2503919be55 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:04.703604 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88abb46e-9cf5-46cf-a302-a2503919be55 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:04.703882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88abb46e-9cf5-46cf-a302-a2503919be55 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:04.704665 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88abb46e-9cf5-46cf-a302-a2503919be55 None admin] RBAC: Adding query filter params (name=member) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:04.704974 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88abb46e-9cf5-46cf-a302-a2503919be55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:04.706365 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88abb46e-9cf5-46cf-a302-a2503919be55 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:04.722685 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 168/342] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 18 06:06:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:06:04.724202 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b046dd5a-0f3b-4a92-82e5-9c61c5f42431 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:04.724525 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b046dd5a-0f3b-4a92-82e5-9c61c5f42431 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:04.724815 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b046dd5a-0f3b-4a92-82e5-9c61c5f42431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:04.757218 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-b046dd5a-0f3b-4a92-82e5-9c61c5f42431 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:04.758426 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-b046dd5a-0f3b-4a92-82e5-9c61c5f42431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:04.884566 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e800f46c-6826-44ad-8cbb-168b7ce6b0c7 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:04.897842 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 169/343] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:06:04.908889 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b046dd5a-0f3b-4a92-82e5-9c61c5f42431 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:04.974451 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c3cc7-031f-4386-ad67-d7b9221890aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:04.976034 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c3cc7-031f-4386-ad67-d7b9221890aa None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:04.980661 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c3cc7-031f-4386-ad67-d7b9221890aa None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:04.981451 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c3cc7-031f-4386-ad67-d7b9221890aa None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:04.983027 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0c3cc7-031f-4386-ad67-d7b9221890aa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:04.984140 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0c3cc7-031f-4386-ad67-d7b9221890aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:04.987610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0c3cc7-031f-4386-ad67-d7b9221890aa None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:05.007257 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:05.014318 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 175/344] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:06:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:05.015214 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c3cc7-031f-4386-ad67-d7b9221890aa None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:05.154309 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c3cc7-031f-4386-ad67-d7b9221890aa None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:05.172974 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b046dd5a-0f3b-4a92-82e5-9c61c5f42431 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:05.196724 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27807c-1742-4f51-83f1-90a562480ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:05.197546 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27807c-1742-4f51-83f1-90a562480ffc None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:05.197843 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27807c-1742-4f51-83f1-90a562480ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:05.198191 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27807c-1742-4f51-83f1-90a562480ffc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:05.209661 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d27807c-1742-4f51-83f1-90a562480ffc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:05.218553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-3d27807c-1742-4f51-83f1-90a562480ffc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 06:06:05.218553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:05.218553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:05.218553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:05.218553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:05.218553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:05.218553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:05.218553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:05.218553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:05.218553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:05.218553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:05.218553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:05.218553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:05.218553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:05.218553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:05.218553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:05.219591 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:05.219591 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:05.219591 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:05.219591 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:05.219591 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:05.219591 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:05.219591 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:05.219591 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:05.219591 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:05.219591 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:05.219591 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:05.219591 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:05.219591 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:05.219591 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:05.219591 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:05.219591 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:05.220403 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:05.220403 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:05.220403 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:05.220403 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:05.220403 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:05.220403 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:05.220403 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:05.220403 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:05.220403 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:05.220403 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:05.220403 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 06:06:05.220403 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:05.220403 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 176/345] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:06:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:05.221101 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27807c-1742-4f51-83f1-90a562480ffc None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:05.233911 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:05.300786 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:05.321192 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe9dcc-0542-476d-ab80-8b429619936e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:05.321890 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe9dcc-0542-476d-ab80-8b429619936e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:05.322186 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe9dcc-0542-476d-ab80-8b429619936e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:05.322476 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe9dcc-0542-476d-ab80-8b429619936e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:05.336518 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27807c-1742-4f51-83f1-90a562480ffc None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:05.387818 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66223048-bf39-4e6b-a5e0-9a6925b98873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:05.388580 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66223048-bf39-4e6b-a5e0-9a6925b98873 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:05.389110 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66223048-bf39-4e6b-a5e0-9a6925b98873 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:05.389400 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66223048-bf39-4e6b-a5e0-9a6925b98873 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:05.445999 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe9dcc-0542-476d-ab80-8b429619936e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:05.450916 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe9dcc-0542-476d-ab80-8b429619936e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:05.482397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66223048-bf39-4e6b-a5e0-9a6925b98873 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:05.484472 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66223048-bf39-4e6b-a5e0-9a6925b98873 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:05.591274 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 170/346] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:06:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 683 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 06:06:05.592312 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe9dcc-0542-476d-ab80-8b429619936e None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:05.609826 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:05.612213 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:05.693581 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 177/347] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:06:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 06:06:05.694706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66223048-bf39-4e6b-a5e0-9a6925b98873 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:05.720326 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe9dcc-0542-476d-ab80-8b429619936e None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:05.764225 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19433481-e5f3-4b24-ae8b-7027b9f943fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:05.765060 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19433481-e5f3-4b24-ae8b-7027b9f943fc None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:05.765415 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19433481-e5f3-4b24-ae8b-7027b9f943fc None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:05.765758 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19433481-e5f3-4b24-ae8b-7027b9f943fc None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:05.766876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19433481-e5f3-4b24-ae8b-7027b9f943fc None admin] RBAC: Adding query filter params (name=member) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:05.767247 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19433481-e5f3-4b24-ae8b-7027b9f943fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:05.770545 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19433481-e5f3-4b24-ae8b-7027b9f943fc None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:05.784590 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 171/348] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 18 06:06:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:05.785731 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19433481-e5f3-4b24-ae8b-7027b9f943fc None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:05.838592 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66223048-bf39-4e6b-a5e0-9a6925b98873 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:05.897989 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:05.898208 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8160bef7-e02a-409b-87c0-6b3bfbcbfa89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:05.899122 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8160bef7-e02a-409b-87c0-6b3bfbcbfa89 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:05.899631 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8160bef7-e02a-409b-87c0-6b3bfbcbfa89 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:05.900102 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8160bef7-e02a-409b-87c0-6b3bfbcbfa89 None admin] PATH_INFO: `/v3/projects/4620d11733d2401eb2122d406221cf0d` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:05.902481 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8160bef7-e02a-409b-87c0-6b3bfbcbfa89 None admin] RBAC: Authorizing `identity:get_project(project_id=4620d11733d2401eb2122d406221cf0d)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:05.904305 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8160bef7-e02a-409b-87c0-6b3bfbcbfa89 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:05.906485 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 178/349] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:06:05 2022] GET /identity/v3/projects/4620d11733d2401eb2122d406221cf0d => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:05.907861 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b5f23-ead9-42cf-956d-cb96f61d91f4 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:05.912688 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b5f23-ead9-42cf-956d-cb96f61d91f4 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:05.927780 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b5f23-ead9-42cf-956d-cb96f61d91f4 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:05.927780 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 179/350] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:05.927780 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3b5f23-ead9-42cf-956d-cb96f61d91f4 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:05.928047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19433481-e5f3-4b24-ae8b-7027b9f943fc None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:05.955874 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:05.971844 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b140e5c-26fe-4379-9f29-61b35bb107e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:05.972665 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b140e5c-26fe-4379-9f29-61b35bb107e6 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:05.972948 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b140e5c-26fe-4379-9f29-61b35bb107e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:05.973225 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b140e5c-26fe-4379-9f29-61b35bb107e6 None admin] PATH_INFO: `/v3/projects/4620d11733d2401eb2122d406221cf0d` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:05.975511 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b140e5c-26fe-4379-9f29-61b35bb107e6 None admin] RBAC: Authorizing `identity:get_project(project_id=4620d11733d2401eb2122d406221cf0d)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:05.977600 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b140e5c-26fe-4379-9f29-61b35bb107e6 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:05.979852 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 172/351] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:06:05 2022] GET /identity/v3/projects/4620d11733d2401eb2122d406221cf0d => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:05.981655 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e885bf-9df6-4600-b78c-766badda2377 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:05.981943 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e885bf-9df6-4600-b78c-766badda2377 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:05.982225 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e885bf-9df6-4600-b78c-766badda2377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:06.005093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-98e885bf-9df6-4600-b78c-766badda2377 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:06.006944 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-98e885bf-9df6-4600-b78c-766badda2377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:06.063702 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3b5f23-ead9-42cf-956d-cb96f61d91f4 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:06.098311 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 173/352] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:06:06.101563 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e885bf-9df6-4600-b78c-766badda2377 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:06.105553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58babf94-b9ee-4b35-820d-450176f6edae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:06.106575 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58babf94-b9ee-4b35-820d-450176f6edae None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:06.107083 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58babf94-b9ee-4b35-820d-450176f6edae None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:06.107556 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58babf94-b9ee-4b35-820d-450176f6edae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:06.182490 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:06.193797 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58babf94-b9ee-4b35-820d-450176f6edae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:06.195680 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58babf94-b9ee-4b35-820d-450176f6edae None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:06.209883 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e885bf-9df6-4600-b78c-766badda2377 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:06.236974 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:06.262783 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f11b0-d135-45c6-aa8d-4c6060c061a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:06.264045 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f11b0-d135-45c6-aa8d-4c6060c061a9 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:06.264340 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f11b0-d135-45c6-aa8d-4c6060c061a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:06.265162 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f11b0-d135-45c6-aa8d-4c6060c061a9 None admin] PATH_INFO: `/v3/groups/b411ad5819cf405088a4733141a3b702` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:06.272679 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f11b0-d135-45c6-aa8d-4c6060c061a9 None admin] RBAC: Authorizing `identity:get_group(group_id=b411ad5819cf405088a4733141a3b702)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:06.274362 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f11b0-d135-45c6-aa8d-4c6060c061a9 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:06.277166 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 174/353] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Jul 18 06:06:06 2022] GET /identity/v3/groups/b411ad5819cf405088a4733141a3b702 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:06.278177 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f11b0-d135-45c6-aa8d-4c6060c061a9 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:06.329238 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 180/354] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:06:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 06:06:06.331206 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58babf94-b9ee-4b35-820d-450176f6edae None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:06.405178 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f11b0-d135-45c6-aa8d-4c6060c061a9 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:06.444303 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58babf94-b9ee-4b35-820d-450176f6edae None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:06.445729 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673de2ac-54cc-4ae7-bf48-2c50cee1e79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:06.446655 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673de2ac-54cc-4ae7-bf48-2c50cee1e79f None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:06.447003 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673de2ac-54cc-4ae7-bf48-2c50cee1e79f None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:06.447364 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673de2ac-54cc-4ae7-bf48-2c50cee1e79f None admin] PATH_INFO: `/v3/groups/b411ad5819cf405088a4733141a3b702` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:06.449423 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673de2ac-54cc-4ae7-bf48-2c50cee1e79f None admin] RBAC: Authorizing `identity:get_group(group_id=b411ad5819cf405088a4733141a3b702)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:06.450899 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673de2ac-54cc-4ae7-bf48-2c50cee1e79f None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:06.452569 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 175/355] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Jul 18 06:06:06 2022] GET /identity/v3/groups/b411ad5819cf405088a4733141a3b702 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:06.454134 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacaa35b-3551-48b7-9b0a-001393d39606 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:06.454480 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacaa35b-3551-48b7-9b0a-001393d39606 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:06.454763 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacaa35b-3551-48b7-9b0a-001393d39606 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:06.456139 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 176/356] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:06.456780 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacaa35b-3551-48b7-9b0a-001393d39606 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:06.468954 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:06.486127 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:06.496696 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd0a6f-8f5d-462d-bfa7-82261251a06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:06.497315 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd0a6f-8f5d-462d-bfa7-82261251a06b None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:06.497566 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd0a6f-8f5d-462d-bfa7-82261251a06b None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:06.497840 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd0a6f-8f5d-462d-bfa7-82261251a06b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:06.509807 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cd0a6f-8f5d-462d-bfa7-82261251a06b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:06.518478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-31cd0a6f-8f5d-462d-bfa7-82261251a06b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 06:06:06.518478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:06.518478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:06.518478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:06.518478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:06.518478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:06.518478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:06.518478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:06.518478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:06.518478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:06.518478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:06.518478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:06.518478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:06.518478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:06.518478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:06.518478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:06.522894 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:06.522894 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:06.522894 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:06.522894 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:06.522894 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:06.522894 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:06.522894 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:06.522894 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:06.522894 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:06.522894 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:06.522894 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:06.522894 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:06.522894 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:06.522894 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:06.522894 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:06.522894 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:06.523691 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:06.523691 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:06.523691 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:06.523691 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:06.523691 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:06.523691 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:06.523691 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:06.523691 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:06.523691 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:06.523691 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:06.523691 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 06:06:06.523691 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:06.523691 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 181/357] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 18 06:06:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:06.523691 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd0a6f-8f5d-462d-bfa7-82261251a06b None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:06.597451 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacaa35b-3551-48b7-9b0a-001393d39606 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:06.636965 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e34f2-4e3d-4814-be51-60c55176e59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:06.637629 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e34f2-4e3d-4814-be51-60c55176e59b None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:06.637913 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e34f2-4e3d-4814-be51-60c55176e59b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:06.638191 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e34f2-4e3d-4814-be51-60c55176e59b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:06.639037 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e34f2-4e3d-4814-be51-60c55176e59b None admin] RBAC: Adding query filter params (group.id=b411ad5819cf405088a4733141a3b702, role.id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20, scope.project.id=4620d11733d2401eb2122d406221cf0d) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:06.639347 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e34f2-4e3d-4814-be51-60c55176e59b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:06.640876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e34f2-4e3d-4814-be51-60c55176e59b None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:06.647412 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 177/358] 10.222.0.40 () {56 vars in 1376 bytes} [Mon Jul 18 06:06:06 2022] GET /identity/v3/role_assignments?group.id=b411ad5819cf405088a4733141a3b702&scope.project.id=4620d11733d2401eb2122d406221cf0d&role.id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:06.652460 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc2458-18fe-420d-9129-56799b685215 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:06.652460 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc2458-18fe-420d-9129-56799b685215 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:06.652460 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc2458-18fe-420d-9129-56799b685215 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:06.652460 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 178/359] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:06.652460 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc2458-18fe-420d-9129-56799b685215 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:06.663350 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd0a6f-8f5d-462d-bfa7-82261251a06b None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:06.714517 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5aba1-ea5a-4720-8033-80a43f8a329b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:06.715669 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5aba1-ea5a-4720-8033-80a43f8a329b None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:06.716085 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5aba1-ea5a-4720-8033-80a43f8a329b None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:06.716505 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5aba1-ea5a-4720-8033-80a43f8a329b None admin] PATH_INFO: `/v3/projects/d34db96dd82a425d90973717021e4c95` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:06.718025 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5aba1-ea5a-4720-8033-80a43f8a329b None admin] RBAC: Authorizing `identity:get_project(project_id=d34db96dd82a425d90973717021e4c95)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:06.719886 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5aba1-ea5a-4720-8033-80a43f8a329b None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:06.721855 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 182/360] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:06:06 2022] GET /identity/v3/projects/d34db96dd82a425d90973717021e4c95 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:06.725886 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3c5ed-ad84-4ab4-a4ed-7f5f78e6c2fd None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:06.726238 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3c5ed-ad84-4ab4-a4ed-7f5f78e6c2fd None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:06.726531 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3c5ed-ad84-4ab4-a4ed-7f5f78e6c2fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:06.748386 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-dac3c5ed-ad84-4ab4-a4ed-7f5f78e6c2fd None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:06.749569 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-dac3c5ed-ad84-4ab4-a4ed-7f5f78e6c2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:06.755528 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:06.800506 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc2458-18fe-420d-9129-56799b685215 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:06.829828 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:06.852699 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 183/361] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:06:06.853813 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3c5ed-ad84-4ab4-a4ed-7f5f78e6c2fd None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:06.855279 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9cad3f-9144-486a-b0ac-237f30d975ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:06.856179 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9cad3f-9144-486a-b0ac-237f30d975ba None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:06.857098 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9cad3f-9144-486a-b0ac-237f30d975ba None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:06.857781 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9cad3f-9144-486a-b0ac-237f30d975ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:06.859765 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9cad3f-9144-486a-b0ac-237f30d975ba None admin] RBAC: Adding query filter params (group.id=b411ad5819cf405088a4733141a3b702, role.id=04868303225144c389068779288f3a42, scope.project.id=4620d11733d2401eb2122d406221cf0d) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:06.860292 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9cad3f-9144-486a-b0ac-237f30d975ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:06.861957 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9cad3f-9144-486a-b0ac-237f30d975ba None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:06.871085 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 179/362] 10.222.0.40 () {56 vars in 1376 bytes} [Mon Jul 18 06:06:06 2022] GET /identity/v3/role_assignments?group.id=b411ad5819cf405088a4733141a3b702&scope.project.id=4620d11733d2401eb2122d406221cf0d&role.id=04868303225144c389068779288f3a42 => generated 260 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:06.873408 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd575ace-8f99-463d-bb34-718470a183a1 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:06.874197 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd575ace-8f99-463d-bb34-718470a183a1 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:06.874666 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd575ace-8f99-463d-bb34-718470a183a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:06.902283 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-cd575ace-8f99-463d-bb34-718470a183a1 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:06.904884 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-cd575ace-8f99-463d-bb34-718470a183a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:06.981652 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3c5ed-ad84-4ab4-a4ed-7f5f78e6c2fd None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:07.006745 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 180/363] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:06:07.007889 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd575ace-8f99-463d-bb34-718470a183a1 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:07.044457 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266977e7-5775-45bb-b937-2c2c439a4cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:07.045505 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266977e7-5775-45bb-b937-2c2c439a4cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:07.045983 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266977e7-5775-45bb-b937-2c2c439a4cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:07.046359 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266977e7-5775-45bb-b937-2c2c439a4cf6 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:07.047251 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266977e7-5775-45bb-b937-2c2c439a4cf6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:07.047667 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266977e7-5775-45bb-b937-2c2c439a4cf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:07.049254 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266977e7-5775-45bb-b937-2c2c439a4cf6 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:07.062651 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 184/364] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:06:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:07.063952 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266977e7-5775-45bb-b937-2c2c439a4cf6 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:07.100107 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:07.128628 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:07.153584 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd575ace-8f99-463d-bb34-718470a183a1 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:07.204078 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266977e7-5775-45bb-b937-2c2c439a4cf6 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:07.209735 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fdf5bb-9a3e-4b04-a9d7-3fd84175e2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:07.211206 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fdf5bb-9a3e-4b04-a9d7-3fd84175e2f0 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:07.212814 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fdf5bb-9a3e-4b04-a9d7-3fd84175e2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:07.213191 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fdf5bb-9a3e-4b04-a9d7-3fd84175e2f0 None admin] PATH_INFO: `/v3/groups/b411ad5819cf405088a4733141a3b702` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:07.215352 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fdf5bb-9a3e-4b04-a9d7-3fd84175e2f0 None admin] RBAC: Authorizing `identity:get_group(group_id=b411ad5819cf405088a4733141a3b702)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:07.216992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fdf5bb-9a3e-4b04-a9d7-3fd84175e2f0 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:07.218844 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 181/365] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Jul 18 06:06:07 2022] GET /identity/v3/groups/b411ad5819cf405088a4733141a3b702 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:07.219698 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fdf5bb-9a3e-4b04-a9d7-3fd84175e2f0 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:07.257106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec9c92-86d8-491e-b0b7-4979414b56c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:07.258267 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec9c92-86d8-491e-b0b7-4979414b56c9 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:07.258754 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec9c92-86d8-491e-b0b7-4979414b56c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:07.259425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec9c92-86d8-491e-b0b7-4979414b56c9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:07.274541 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bec9c92-86d8-491e-b0b7-4979414b56c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:07.284609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-3bec9c92-86d8-491e-b0b7-4979414b56c9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 06:06:07.284609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:07.284609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:07.284609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:07.284609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:07.284609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:07.284609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:07.284609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:07.284609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:07.284609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:07.284609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:07.284609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:07.284609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:07.284609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:07.284609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:07.284609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:07.285599 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:07.285599 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:07.285599 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:07.285599 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:07.285599 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:07.285599 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:07.285599 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:07.285599 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:07.285599 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:07.285599 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:07.285599 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:07.285599 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:07.285599 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:07.285599 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:07.285599 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:07.285599 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:07.286421 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:07.286421 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:07.286421 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:07.286421 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:07.286421 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:07.286421 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:07.286421 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:07.286421 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:07.286421 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:07.286421 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:07.286421 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 06:06:07.286421 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:07.290292 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 185/366] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:06:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:07.291407 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec9c92-86d8-491e-b0b7-4979414b56c9 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:07.362989 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fdf5bb-9a3e-4b04-a9d7-3fd84175e2f0 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:07.403621 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:07.409261 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e24fb-8273-4c17-afa3-3e404078db83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:07.410366 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e24fb-8273-4c17-afa3-3e404078db83 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:07.410891 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e24fb-8273-4c17-afa3-3e404078db83 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:07.412747 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e24fb-8273-4c17-afa3-3e404078db83 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:07.425808 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32e24fb-8273-4c17-afa3-3e404078db83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:07.434993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-c32e24fb-8273-4c17-afa3-3e404078db83 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:06:07.434993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:07.434993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:07.434993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:07.434993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:07.434993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:07.434993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:07.434993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:07.434993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:07.434993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:07.434993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:07.434993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:07.434993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:07.434993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:07.434993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:07.434993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:07.435939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:07.435939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:07.435939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:07.435939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:07.435939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:07.435939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:07.435939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:07.435939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:07.435939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:07.435939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:07.435939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:07.435939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:07.435939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:07.435939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:07.435939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:07.435939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:07.436784 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:07.436784 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:07.436784 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:07.436784 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:07.436784 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:07.436784 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:07.436784 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:07.436784 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:07.436784 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:07.436784 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:07.436784 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:06:07.436784 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:07.440100 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 182/367] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 18 06:06:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:07.441329 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e24fb-8273-4c17-afa3-3e404078db83 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:07.444141 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec9c92-86d8-491e-b0b7-4979414b56c9 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:07.454833 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:07.494773 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d6407-59cd-4404-b666-eec3c9d9503c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:07.496881 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d6407-59cd-4404-b666-eec3c9d9503c None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:07.497272 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d6407-59cd-4404-b666-eec3c9d9503c None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:07.497631 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d6407-59cd-4404-b666-eec3c9d9503c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:07.571608 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e24fb-8273-4c17-afa3-3e404078db83 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:07.586883 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d6407-59cd-4404-b666-eec3c9d9503c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:07.589159 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d6407-59cd-4404-b666-eec3c9d9503c None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:07.612167 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd1b32a-12c7-475f-bb58-7cb5c01f373a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:07.613336 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd1b32a-12c7-475f-bb58-7cb5c01f373a None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:07.613830 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd1b32a-12c7-475f-bb58-7cb5c01f373a None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:07.614376 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd1b32a-12c7-475f-bb58-7cb5c01f373a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:07.615517 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd1b32a-12c7-475f-bb58-7cb5c01f373a None admin] RBAC: Adding query filter params (group.id=b411ad5819cf405088a4733141a3b702, role.id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20, scope.project.id=d34db96dd82a425d90973717021e4c95) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:07.616121 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd1b32a-12c7-475f-bb58-7cb5c01f373a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:07.617820 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd1b32a-12c7-475f-bb58-7cb5c01f373a None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:07.626594 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 183/368] 10.222.0.40 () {56 vars in 1376 bytes} [Mon Jul 18 06:06:07 2022] GET /identity/v3/role_assignments?group.id=b411ad5819cf405088a4733141a3b702&scope.project.id=d34db96dd82a425d90973717021e4c95&role.id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:07.628216 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd1b32a-12c7-475f-bb58-7cb5c01f373a None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:07.696050 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:07.710302 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 186/369] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:06:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 06:06:07.711729 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d6407-59cd-4404-b666-eec3c9d9503c None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:07.724098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:07.756807 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd1b32a-12c7-475f-bb58-7cb5c01f373a None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:07.795453 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540186a5-fd46-4526-8467-4e2565f15e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:07.796332 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540186a5-fd46-4526-8467-4e2565f15e56 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:07.796710 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540186a5-fd46-4526-8467-4e2565f15e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:07.797091 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540186a5-fd46-4526-8467-4e2565f15e56 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:07.798395 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540186a5-fd46-4526-8467-4e2565f15e56 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:07.798395 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540186a5-fd46-4526-8467-4e2565f15e56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:07.799983 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540186a5-fd46-4526-8467-4e2565f15e56 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:07.809710 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 184/370] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 18 06:06:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:07.810744 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540186a5-fd46-4526-8467-4e2565f15e56 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:07.837151 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d6407-59cd-4404-b666-eec3c9d9503c None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:07.878723 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497d8e1-9db5-424f-9771-acea22783ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:07.879726 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497d8e1-9db5-424f-9771-acea22783ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:07.880225 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497d8e1-9db5-424f-9771-acea22783ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:07.880714 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497d8e1-9db5-424f-9771-acea22783ff8 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:07.881682 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4497d8e1-9db5-424f-9771-acea22783ff8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:07.882176 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4497d8e1-9db5-424f-9771-acea22783ff8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:07.883836 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4497d8e1-9db5-424f-9771-acea22783ff8 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:07.894685 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 187/371] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:06:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:07.896009 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497d8e1-9db5-424f-9771-acea22783ff8 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:07.939574 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540186a5-fd46-4526-8467-4e2565f15e56 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:07.961587 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:07.983022 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26481fe-3679-49bb-b98b-a0a604771814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:07.984017 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26481fe-3679-49bb-b98b-a0a604771814 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:07.984380 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26481fe-3679-49bb-b98b-a0a604771814 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:07.985524 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26481fe-3679-49bb-b98b-a0a604771814 None admin] PATH_INFO: `/v3/projects/d34db96dd82a425d90973717021e4c95` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:07.986988 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26481fe-3679-49bb-b98b-a0a604771814 None admin] RBAC: Authorizing `identity:get_project(project_id=d34db96dd82a425d90973717021e4c95)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:07.988654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26481fe-3679-49bb-b98b-a0a604771814 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:07.990482 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 185/372] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:06:07 2022] GET /identity/v3/projects/d34db96dd82a425d90973717021e4c95 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:07.990875 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:07.991529 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26481fe-3679-49bb-b98b-a0a604771814 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:08.033638 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497d8e1-9db5-424f-9771-acea22783ff8 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:08.076632 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698db2c6-d24c-4e8e-ad14-3a19eba609db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:08.078317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698db2c6-d24c-4e8e-ad14-3a19eba609db None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:08.078799 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698db2c6-d24c-4e8e-ad14-3a19eba609db None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:08.079257 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698db2c6-d24c-4e8e-ad14-3a19eba609db None admin] PATH_INFO: `/v3/users/f4524ff1ba3344b9b506631806564d55` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:08.081033 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698db2c6-d24c-4e8e-ad14-3a19eba609db None admin] RBAC: Authorizing `identity:get_user(user_id=f4524ff1ba3344b9b506631806564d55)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:08.082822 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698db2c6-d24c-4e8e-ad14-3a19eba609db None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:08.085344 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 188/373] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:06:07 2022] GET /identity/v3/users/f4524ff1ba3344b9b506631806564d55 => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:08.086352 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698db2c6-d24c-4e8e-ad14-3a19eba609db None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:08.114935 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26481fe-3679-49bb-b98b-a0a604771814 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:08.152003 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98051ce0-f7d7-4715-8363-6cd4c0c1a130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:08.152823 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98051ce0-f7d7-4715-8363-6cd4c0c1a130 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:08.153207 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98051ce0-f7d7-4715-8363-6cd4c0c1a130 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:08.153566 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98051ce0-f7d7-4715-8363-6cd4c0c1a130 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:08.204856 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698db2c6-d24c-4e8e-ad14-3a19eba609db None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:08.216037 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:08.230150 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98051ce0-f7d7-4715-8363-6cd4c0c1a130 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:08.231723 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98051ce0-f7d7-4715-8363-6cd4c0c1a130 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:08.247642 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5665bbb1-436a-4307-ac56-62eb19a33d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:08.247642 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665bbb1-436a-4307-ac56-62eb19a33d1d None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:08.247642 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665bbb1-436a-4307-ac56-62eb19a33d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:08.247642 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665bbb1-436a-4307-ac56-62eb19a33d1d None admin] PATH_INFO: `/v3/groups/b411ad5819cf405088a4733141a3b702` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:08.248967 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5665bbb1-436a-4307-ac56-62eb19a33d1d None admin] RBAC: Authorizing `identity:get_group(group_id=b411ad5819cf405088a4733141a3b702)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:08.250632 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5665bbb1-436a-4307-ac56-62eb19a33d1d None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:08.252515 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 189/374] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Jul 18 06:06:08 2022] GET /identity/v3/groups/b411ad5819cf405088a4733141a3b702 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:08.253565 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5665bbb1-436a-4307-ac56-62eb19a33d1d None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:08.264101 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:08.346062 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 186/375] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:06:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 06:06:08.346830 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98051ce0-f7d7-4715-8363-6cd4c0c1a130 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:08.373153 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5665bbb1-436a-4307-ac56-62eb19a33d1d None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:08.411884 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dec505-dd85-48ca-ac45-d2d7eaafd0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:08.413252 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dec505-dd85-48ca-ac45-d2d7eaafd0dd None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:08.413726 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dec505-dd85-48ca-ac45-d2d7eaafd0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:08.414187 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dec505-dd85-48ca-ac45-d2d7eaafd0dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:08.415329 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dec505-dd85-48ca-ac45-d2d7eaafd0dd None admin] RBAC: Adding query filter params (role.id=432bfb33f0224d11acc5bae2c196dc77, user.id=f4524ff1ba3344b9b506631806564d55, scope.system=all) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:08.415841 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dec505-dd85-48ca-ac45-d2d7eaafd0dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:08.418385 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dec505-dd85-48ca-ac45-d2d7eaafd0dd None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:08.433083 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 190/376] 10.222.0.40 () {56 vars in 1308 bytes} [Mon Jul 18 06:06:08 2022] GET /identity/v3/role_assignments?user.id=f4524ff1ba3344b9b506631806564d55&scope.system=all&role.id=432bfb33f0224d11acc5bae2c196dc77 => generated 510 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:08.434495 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dec505-dd85-48ca-ac45-d2d7eaafd0dd None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:08.445869 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:08.467416 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98051ce0-f7d7-4715-8363-6cd4c0c1a130 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:08.519684 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:08.522667 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d1d50-b616-482d-a816-9b3002efb83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:08.523424 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d1d50-b616-482d-a816-9b3002efb83e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:08.523781 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d1d50-b616-482d-a816-9b3002efb83e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:08.524134 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d1d50-b616-482d-a816-9b3002efb83e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:08.525114 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3d1d50-b616-482d-a816-9b3002efb83e None admin] RBAC: Adding query filter params (group.id=b411ad5819cf405088a4733141a3b702, role.id=04868303225144c389068779288f3a42, scope.project.id=d34db96dd82a425d90973717021e4c95) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:08.525499 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3d1d50-b616-482d-a816-9b3002efb83e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:08.532454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3d1d50-b616-482d-a816-9b3002efb83e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:08.535799 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 187/377] 10.222.0.40 () {56 vars in 1376 bytes} [Mon Jul 18 06:06:08 2022] GET /identity/v3/role_assignments?group.id=b411ad5819cf405088a4733141a3b702&scope.project.id=d34db96dd82a425d90973717021e4c95&role.id=04868303225144c389068779288f3a42 => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:08.579710 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dec505-dd85-48ca-ac45-d2d7eaafd0dd None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:08.624299 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4214b2cc-4edc-49cc-a7ec-10ac8b0b67cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:08.625150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214b2cc-4edc-49cc-a7ec-10ac8b0b67cf None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:08.625435 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214b2cc-4edc-49cc-a7ec-10ac8b0b67cf None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:08.625715 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214b2cc-4edc-49cc-a7ec-10ac8b0b67cf None admin] PATH_INFO: `/v3/projects/3e19cd5e83fe45f5a260277527d6bf8c` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:08.627056 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4214b2cc-4edc-49cc-a7ec-10ac8b0b67cf None admin] RBAC: Authorizing `identity:get_project(project_id=3e19cd5e83fe45f5a260277527d6bf8c)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:08.628524 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4214b2cc-4edc-49cc-a7ec-10ac8b0b67cf None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:08.630615 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 191/378] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:06:08 2022] GET /identity/v3/projects/3e19cd5e83fe45f5a260277527d6bf8c => generated 344 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:08.636747 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d1d50-b616-482d-a816-9b3002efb83e None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:08.779831 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d1d50-b616-482d-a816-9b3002efb83e None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:08.824202 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5778dc04-63a8-40c7-9805-f22423d25d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:08.825122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5778dc04-63a8-40c7-9805-f22423d25d52 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:08.825417 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5778dc04-63a8-40c7-9805-f22423d25d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:08.825694 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5778dc04-63a8-40c7-9805-f22423d25d52 None admin] PATH_INFO: `/v3/groups/f3f6c1340437475382e7c75855b0124c` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:08.831588 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5778dc04-63a8-40c7-9805-f22423d25d52 None admin] RBAC: Authorizing `identity:get_group(group_id=f3f6c1340437475382e7c75855b0124c)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:08.833153 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5778dc04-63a8-40c7-9805-f22423d25d52 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:08.835277 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 188/379] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Jul 18 06:06:08 2022] GET /identity/v3/groups/f3f6c1340437475382e7c75855b0124c => generated 229 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:08.841404 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4214b2cc-4edc-49cc-a7ec-10ac8b0b67cf None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:08.964242 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4214b2cc-4edc-49cc-a7ec-10ac8b0b67cf None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:08.996718 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:09.005369 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973746a-ca0a-45ef-a030-aaffb503fcd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:09.006039 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973746a-ca0a-45ef-a030-aaffb503fcd5 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:09.006325 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973746a-ca0a-45ef-a030-aaffb503fcd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:09.006604 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973746a-ca0a-45ef-a030-aaffb503fcd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:09.007457 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2973746a-ca0a-45ef-a030-aaffb503fcd5 None admin] RBAC: Adding query filter params (group.id=f3f6c1340437475382e7c75855b0124c, role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc, scope.project.id=3e19cd5e83fe45f5a260277527d6bf8c) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:09.007777 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2973746a-ca0a-45ef-a030-aaffb503fcd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:09.009243 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2973746a-ca0a-45ef-a030-aaffb503fcd5 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:09.017020 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 192/380] 10.222.0.40 () {56 vars in 1376 bytes} [Mon Jul 18 06:06:08 2022] GET /identity/v3/role_assignments?group.id=f3f6c1340437475382e7c75855b0124c&scope.project.id=3e19cd5e83fe45f5a260277527d6bf8c&role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:09.286429 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015eefde-a77e-438c-b8aa-9b47792f9d30 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:09.286910 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015eefde-a77e-438c-b8aa-9b47792f9d30 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:09.286910 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015eefde-a77e-438c-b8aa-9b47792f9d30 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:09.288869 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 189/381] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:09.295172 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036fc5e5-0132-4f35-bff8-a60140e818b6 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:09.295636 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036fc5e5-0132-4f35-bff8-a60140e818b6 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:09.295714 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036fc5e5-0132-4f35-bff8-a60140e818b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:09.312724 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-036fc5e5-0132-4f35-bff8-a60140e818b6 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:09.313922 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-036fc5e5-0132-4f35-bff8-a60140e818b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:09.402772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 193/382] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:06:09.442081 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015eefde-a77e-438c-b8aa-9b47792f9d30 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:09.459553 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:09.578255 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015eefde-a77e-438c-b8aa-9b47792f9d30 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:09.624406 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a26ea36-277c-4e36-bd8c-bac4d102c7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:09.625271 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26ea36-277c-4e36-bd8c-bac4d102c7bc None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:09.625555 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26ea36-277c-4e36-bd8c-bac4d102c7bc None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:09.625836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26ea36-277c-4e36-bd8c-bac4d102c7bc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:09.646438 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a26ea36-277c-4e36-bd8c-bac4d102c7bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:09.655180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-2a26ea36-277c-4e36-bd8c-bac4d102c7bc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 06:06:09.655180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:09.655180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:09.655180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:09.655180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:09.655180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:09.655180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:09.655180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:09.655180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:09.655180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:09.655180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:09.655180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:09.655180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:09.655180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:09.655180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:09.655180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:09.656112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:09.656112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:09.656112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:09.656112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:09.656112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:09.656112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:09.656112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:09.656112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:09.656112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:09.656112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:09.656112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:09.656112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:09.656112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:09.656112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:09.656112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:09.656112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:09.656971 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:09.656971 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:09.656971 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:09.656971 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:09.656971 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:09.656971 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:09.656971 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:09.656971 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:09.656971 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:09.656971 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:09.656971 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 06:06:09.656971 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:09.660894 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 190/383] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:06:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:09.667357 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036fc5e5-0132-4f35-bff8-a60140e818b6 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:09.788783 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036fc5e5-0132-4f35-bff8-a60140e818b6 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:09.818281 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:09.828806 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f7f7e-df45-4ebd-9706-b921aebc1b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:09.829421 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f7f7e-df45-4ebd-9706-b921aebc1b66 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:09.829709 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f7f7e-df45-4ebd-9706-b921aebc1b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:09.830018 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f7f7e-df45-4ebd-9706-b921aebc1b66 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:09.830745 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88f7f7e-df45-4ebd-9706-b921aebc1b66 None admin] RBAC: Adding query filter params (name=member) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:09.831055 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88f7f7e-df45-4ebd-9706-b921aebc1b66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:09.832448 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88f7f7e-df45-4ebd-9706-b921aebc1b66 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:09.841861 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 194/384] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 18 06:06:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:09.847633 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a26ea36-277c-4e36-bd8c-bac4d102c7bc None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:09.954088 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:09.985543 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a26ea36-277c-4e36-bd8c-bac4d102c7bc None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:10.035545 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a31978-fc45-41de-ada2-0d067be9a572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:10.036451 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a31978-fc45-41de-ada2-0d067be9a572 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:10.036743 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a31978-fc45-41de-ada2-0d067be9a572 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:10.037021 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a31978-fc45-41de-ada2-0d067be9a572 None admin] PATH_INFO: `/v3/groups/b411ad5819cf405088a4733141a3b702` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:10.038649 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a31978-fc45-41de-ada2-0d067be9a572 None admin] RBAC: Authorizing `identity:get_group(group_id=b411ad5819cf405088a4733141a3b702)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:10.040170 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a31978-fc45-41de-ada2-0d067be9a572 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:10.042338 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 191/385] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Jul 18 06:06:09 2022] GET /identity/v3/groups/b411ad5819cf405088a4733141a3b702 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:06:10.048080 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f7f7e-df45-4ebd-9706-b921aebc1b66 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:10.167995 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f7f7e-df45-4ebd-9706-b921aebc1b66 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:10.193158 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e752bb6-c35b-47d3-8951-fb182a09a929 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:10.193480 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e752bb6-c35b-47d3-8951-fb182a09a929 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:10.193750 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e752bb6-c35b-47d3-8951-fb182a09a929 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:10.195137 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 192/386] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:06:10.209410 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92950cf-f085-46c8-8a3a-6e13db211ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:10.210284 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92950cf-f085-46c8-8a3a-6e13db211ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:10.210567 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92950cf-f085-46c8-8a3a-6e13db211ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:10.210846 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92950cf-f085-46c8-8a3a-6e13db211ef1 None admin] PATH_INFO: `/v3/projects/4620d11733d2401eb2122d406221cf0d` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:10.213122 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92950cf-f085-46c8-8a3a-6e13db211ef1 None admin] RBAC: Authorizing `identity:get_project(project_id=4620d11733d2401eb2122d406221cf0d)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:10.214642 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b93f9-7c6a-4f86-82f7-2a7c2d5f2943 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:10.215067 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92950cf-f085-46c8-8a3a-6e13db211ef1 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:10.215361 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b93f9-7c6a-4f86-82f7-2a7c2d5f2943 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:10.215853 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b93f9-7c6a-4f86-82f7-2a7c2d5f2943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:10.217196 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 195/387] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:06:10 2022] GET /identity/v3/projects/4620d11733d2401eb2122d406221cf0d => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:10.225505 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92950cf-f085-46c8-8a3a-6e13db211ef1 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:10.243213 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-881b93f9-7c6a-4f86-82f7-2a7c2d5f2943 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:10.245632 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-881b93f9-7c6a-4f86-82f7-2a7c2d5f2943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:10.341358 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:10.378385 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 193/388] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:06:10.405143 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92950cf-f085-46c8-8a3a-6e13db211ef1 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:10.415252 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b93f9-7c6a-4f86-82f7-2a7c2d5f2943 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:10.469847 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92846666-2f0b-4742-a692-8381167e9e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:10.471258 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92846666-2f0b-4742-a692-8381167e9e91 None admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:10.471740 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92846666-2f0b-4742-a692-8381167e9e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:10.472243 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92846666-2f0b-4742-a692-8381167e9e91 None admin] PATH_INFO: `/v3/projects/4620d11733d2401eb2122d406221cf0d/groups/b411ad5819cf405088a4733141a3b702/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:10.475475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92846666-2f0b-4742-a692-8381167e9e91 None admin] RBAC: Authorizing `identity:create_grant(project_id=4620d11733d2401eb2122d406221cf0d, group_id=b411ad5819cf405088a4733141a3b702, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:10.477174 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92846666-2f0b-4742-a692-8381167e9e91 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:10.534805 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:10.566216 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b93f9-7c6a-4f86-82f7-2a7c2d5f2943 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:10.581285 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 196/389] 10.222.0.40 () {58 vars in 1442 bytes} [Mon Jul 18 06:06:10 2022] PUT /identity/v3/projects/4620d11733d2401eb2122d406221cf0d/groups/b411ad5819cf405088a4733141a3b702/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:06:10.617078 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd15f0-ede2-4651-b4ed-19f5948f4280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:10.618297 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd15f0-ede2-4651-b4ed-19f5948f4280 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:10.618810 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd15f0-ede2-4651-b4ed-19f5948f4280 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:10.619143 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd15f0-ede2-4651-b4ed-19f5948f4280 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:10.633552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfd15f0-ede2-4651-b4ed-19f5948f4280 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:10.642657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-4dfd15f0-ede2-4651-b4ed-19f5948f4280 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 06:06:10.642657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:10.642657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:10.642657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:10.642657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:10.642657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:10.642657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:10.642657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:10.642657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:10.642657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:10.642657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:10.642657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:10.642657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:10.642657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:10.642657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:10.642657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:10.643662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:10.643662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:10.643662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:10.643662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:10.643662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:10.643662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:10.643662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:10.643662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:10.643662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:10.643662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:10.643662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:10.643662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:10.643662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:10.643662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:10.643662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:10.643662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:10.644633 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:10.644633 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:10.644633 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:10.644633 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:10.644633 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:10.644633 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:10.644633 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:10.644633 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:10.644633 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:10.644633 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:10.644633 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 06:06:10.644633 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:10.644633 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 194/390] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 18 06:06:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:06:10.650326 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92846666-2f0b-4742-a692-8381167e9e91 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:10.777496 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92846666-2f0b-4742-a692-8381167e9e91 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:10.797403 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:10.817097 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73906120-8741-4524-8add-a031aac3c58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:10.817782 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73906120-8741-4524-8add-a031aac3c58e None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:10.818072 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73906120-8741-4524-8add-a031aac3c58e None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:10.818352 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73906120-8741-4524-8add-a031aac3c58e None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:10.819125 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73906120-8741-4524-8add-a031aac3c58e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:10.819436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73906120-8741-4524-8add-a031aac3c58e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:10.820897 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73906120-8741-4524-8add-a031aac3c58e None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:10.831485 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 197/391] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:06:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:10.837226 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd15f0-ede2-4651-b4ed-19f5948f4280 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:10.955972 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd15f0-ede2-4651-b4ed-19f5948f4280 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:10.991566 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:10.993823 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c653db9-db6f-49c8-8456-06b34d4308ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:10.994760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c653db9-db6f-49c8-8456-06b34d4308ca None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:10.995104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c653db9-db6f-49c8-8456-06b34d4308ca None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:10.995339 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c653db9-db6f-49c8-8456-06b34d4308ca None admin] PATH_INFO: `/v3/groups/b411ad5819cf405088a4733141a3b702` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:10.996795 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c653db9-db6f-49c8-8456-06b34d4308ca None admin] RBAC: Authorizing `identity:get_group(group_id=b411ad5819cf405088a4733141a3b702)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:10.998274 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c653db9-db6f-49c8-8456-06b34d4308ca None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:11.000233 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 195/392] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Jul 18 06:06:10 2022] GET /identity/v3/groups/b411ad5819cf405088a4733141a3b702 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:11.005930 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73906120-8741-4524-8add-a031aac3c58e None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:11.085528 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21893c-706d-4c37-97a7-07c51f73e048 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:11.085806 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21893c-706d-4c37-97a7-07c51f73e048 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:11.086019 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21893c-706d-4c37-97a7-07c51f73e048 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:11.087722 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 196/393] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:06:11.095672 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a1165-6c34-4d66-acd2-6672c4c1e2b4 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:11.096114 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a1165-6c34-4d66-acd2-6672c4c1e2b4 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:11.096530 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a1165-6c34-4d66-acd2-6672c4c1e2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:11.119915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-365a1165-6c34-4d66-acd2-6672c4c1e2b4 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:11.121525 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-365a1165-6c34-4d66-acd2-6672c4c1e2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:11.123996 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73906120-8741-4524-8add-a031aac3c58e None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:11.202074 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee6f24-56d2-4fcc-ae92-f8baf3207993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:11.202975 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee6f24-56d2-4fcc-ae92-f8baf3207993 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:11.203266 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee6f24-56d2-4fcc-ae92-f8baf3207993 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:11.203547 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee6f24-56d2-4fcc-ae92-f8baf3207993 None admin] PATH_INFO: `/v3/projects/4620d11733d2401eb2122d406221cf0d` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:11.205024 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ee6f24-56d2-4fcc-ae92-f8baf3207993 None admin] RBAC: Authorizing `identity:get_project(project_id=4620d11733d2401eb2122d406221cf0d)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:11.206472 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ee6f24-56d2-4fcc-ae92-f8baf3207993 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:11.208218 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 198/394] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:06:11 2022] GET /identity/v3/projects/4620d11733d2401eb2122d406221cf0d => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:11.215059 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee6f24-56d2-4fcc-ae92-f8baf3207993 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:11.237880 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 197/395] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:06:11.285365 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a1165-6c34-4d66-acd2-6672c4c1e2b4 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:11.304884 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:11.366920 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee6f24-56d2-4fcc-ae92-f8baf3207993 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:11.399356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a1165-6c34-4d66-acd2-6672c4c1e2b4 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:11.404595 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1c73d8-88c9-4961-b1bd-4271df0b3fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:11.405494 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c73d8-88c9-4961-b1bd-4271df0b3fa2 None admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:11.405782 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c73d8-88c9-4961-b1bd-4271df0b3fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:11.406066 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c73d8-88c9-4961-b1bd-4271df0b3fa2 None admin] PATH_INFO: `/v3/projects/4620d11733d2401eb2122d406221cf0d/groups/b411ad5819cf405088a4733141a3b702/roles/04868303225144c389068779288f3a42` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:11.408353 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1c73d8-88c9-4961-b1bd-4271df0b3fa2 None admin] RBAC: Authorizing `identity:create_grant(project_id=4620d11733d2401eb2122d406221cf0d, group_id=b411ad5819cf405088a4733141a3b702, role_id=04868303225144c389068779288f3a42)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:11.409925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1c73d8-88c9-4961-b1bd-4271df0b3fa2 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:11.434491 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:11.442467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd73b93-bba5-4044-8bc7-3e9605b39429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:11.443282 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd73b93-bba5-4044-8bc7-3e9605b39429 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:11.443573 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd73b93-bba5-4044-8bc7-3e9605b39429 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:11.443855 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd73b93-bba5-4044-8bc7-3e9605b39429 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:11.455389 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd73b93-bba5-4044-8bc7-3e9605b39429 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:11.463464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-dcd73b93-bba5-4044-8bc7-3e9605b39429 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 06:06:11.463464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:11.463464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:11.463464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:11.463464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:11.463464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:11.463464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:11.463464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:11.463464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:11.463464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:11.463464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:11.463464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:11.463464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:11.463464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:11.463464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:11.463464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:11.464577 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:11.464577 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:11.464577 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:11.464577 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:11.464577 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:11.464577 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:11.464577 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:11.464577 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:11.464577 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:11.464577 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:11.464577 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:11.464577 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:11.464577 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:11.464577 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:11.464577 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:11.464577 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:11.465386 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:11.465386 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:11.465386 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:11.465386 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:11.465386 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:11.465386 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:11.465386 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:11.465386 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:11.465386 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:11.465386 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:11.465386 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 06:06:11.465386 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:11.465386 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 198/396] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:06:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:06:11.471475 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9eafd-0b95-43ae-9975-535c8c32cac8 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:11.471475 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9eafd-0b95-43ae-9975-535c8c32cac8 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:11.471475 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9eafd-0b95-43ae-9975-535c8c32cac8 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:11.471475 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 199/397] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:11.473303 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a9eafd-0b95-43ae-9975-535c8c32cac8 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:11.505063 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 199/398] 10.222.0.40 () {58 vars in 1442 bytes} [Mon Jul 18 06:06:11 2022] PUT /identity/v3/projects/4620d11733d2401eb2122d406221cf0d/groups/b411ad5819cf405088a4733141a3b702/roles/04868303225144c389068779288f3a42 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:06:11.506779 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856deb4-6a79-4b7a-8c0a-b4608bce9fa4 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:11.511290 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856deb4-6a79-4b7a-8c0a-b4608bce9fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:11.511290 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856deb4-6a79-4b7a-8c0a-b4608bce9fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:11.529864 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-5856deb4-6a79-4b7a-8c0a-b4608bce9fa4 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:11.533300 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-5856deb4-6a79-4b7a-8c0a-b4608bce9fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:11.621350 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 200/399] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:06:11.663135 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a9eafd-0b95-43ae-9975-535c8c32cac8 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:11.668022 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5856deb4-6a79-4b7a-8c0a-b4608bce9fa4 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:11.714161 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:11.753264 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c0f8dc-2f32-4be7-b308-f16ba4ab9a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:11.754001 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0f8dc-2f32-4be7-b308-f16ba4ab9a4e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:11.754297 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0f8dc-2f32-4be7-b308-f16ba4ab9a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:11.754585 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0f8dc-2f32-4be7-b308-f16ba4ab9a4e None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:11.758672 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c0f8dc-2f32-4be7-b308-f16ba4ab9a4e None admin] RBAC: Adding query filter params (name=member) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:11.758672 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c0f8dc-2f32-4be7-b308-f16ba4ab9a4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:11.758672 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c0f8dc-2f32-4be7-b308-f16ba4ab9a4e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:11.778016 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 200/400] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 18 06:06:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:11.783967 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c0f8dc-2f32-4be7-b308-f16ba4ab9a4e None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:11.808450 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5856deb4-6a79-4b7a-8c0a-b4608bce9fa4 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:11.845963 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:11.857234 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab5a0d-7fc7-42ce-9c36-a2fa4892dadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:11.858201 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab5a0d-7fc7-42ce-9c36-a2fa4892dadf None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:11.858496 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab5a0d-7fc7-42ce-9c36-a2fa4892dadf None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:11.858785 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab5a0d-7fc7-42ce-9c36-a2fa4892dadf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:11.885331 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ab5a0d-7fc7-42ce-9c36-a2fa4892dadf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:11.896286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-94ab5a0d-7fc7-42ce-9c36-a2fa4892dadf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 06:06:11.896286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:11.896286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:11.896286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:11.896286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:11.896286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:11.896286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:11.896286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:11.896286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:11.896286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:11.896286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:11.896286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:11.896286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:11.896286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:11.896286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:11.896286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:11.897289 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:11.897289 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:11.897289 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:11.897289 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:11.897289 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:11.897289 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:11.897289 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:11.897289 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:11.897289 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:11.897289 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:11.897289 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:11.897289 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:11.897289 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:11.897289 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:11.897289 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:11.897289 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:11.898121 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:11.898121 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:11.898121 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:11.898121 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:11.898121 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:11.898121 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:11.898121 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:11.898121 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:11.898121 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:11.898121 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:11.898121 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 06:06:11.898121 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:11.904448 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 201/401] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 18 06:06:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:11.904448 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab5a0d-7fc7-42ce-9c36-a2fa4892dadf None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:11.915542 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c0f8dc-2f32-4be7-b308-f16ba4ab9a4e None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:11.975590 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ee0de4-68ae-4d04-bf8f-9c385c375ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:11.976499 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee0de4-68ae-4d04-bf8f-9c385c375ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:11.977252 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee0de4-68ae-4d04-bf8f-9c385c375ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:11.977542 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee0de4-68ae-4d04-bf8f-9c385c375ee9 None admin] PATH_INFO: `/v3/groups/b411ad5819cf405088a4733141a3b702` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:11.979632 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ee0de4-68ae-4d04-bf8f-9c385c375ee9 None admin] RBAC: Authorizing `identity:get_group(group_id=b411ad5819cf405088a4733141a3b702)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:11.981672 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ee0de4-68ae-4d04-bf8f-9c385c375ee9 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:11.983997 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 201/402] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Jul 18 06:06:11 2022] GET /identity/v3/groups/b411ad5819cf405088a4733141a3b702 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:11.985539 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a8aed0-c0eb-429d-a967-2fa4ae903e22 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:11.986112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a8aed0-c0eb-429d-a967-2fa4ae903e22 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:11.986393 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a8aed0-c0eb-429d-a967-2fa4ae903e22 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:11.988099 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 202/403] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:11.993429 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a8aed0-c0eb-429d-a967-2fa4ae903e22 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:12.047596 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:12.055981 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab5a0d-7fc7-42ce-9c36-a2fa4892dadf None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:12.095135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519256c6-dda6-45c9-8bd3-173aeb00259b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:12.095912 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519256c6-dda6-45c9-8bd3-173aeb00259b None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:12.096253 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519256c6-dda6-45c9-8bd3-173aeb00259b None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:12.096569 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519256c6-dda6-45c9-8bd3-173aeb00259b None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:12.097338 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519256c6-dda6-45c9-8bd3-173aeb00259b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:12.097702 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519256c6-dda6-45c9-8bd3-173aeb00259b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:12.099196 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519256c6-dda6-45c9-8bd3-173aeb00259b None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:12.112582 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 202/404] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:06:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:06:12.114251 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe4555-9e75-4182-a772-36c41679e5a3 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:12.114711 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe4555-9e75-4182-a772-36c41679e5a3 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:12.115201 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe4555-9e75-4182-a772-36c41679e5a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:12.118729 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a8aed0-c0eb-429d-a967-2fa4ae903e22 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:12.135993 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-9efe4555-9e75-4182-a772-36c41679e5a3 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:12.139377 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-9efe4555-9e75-4182-a772-36c41679e5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:12.148849 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:12.169320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a8284-e2c6-465c-a67e-cc69354c23af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:12.171290 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a8284-e2c6-465c-a67e-cc69354c23af None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:12.172066 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a8284-e2c6-465c-a67e-cc69354c23af None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:12.172612 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a8284-e2c6-465c-a67e-cc69354c23af None admin] PATH_INFO: `/v3/projects/d34db96dd82a425d90973717021e4c95` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:12.175178 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a8284-e2c6-465c-a67e-cc69354c23af None admin] RBAC: Authorizing `identity:get_project(project_id=d34db96dd82a425d90973717021e4c95)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:12.176901 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a8284-e2c6-465c-a67e-cc69354c23af None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:12.185612 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 203/405] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:06:11 2022] GET /identity/v3/projects/d34db96dd82a425d90973717021e4c95 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:12.185612 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a8284-e2c6-465c-a67e-cc69354c23af None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:12.233280 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 203/406] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:06:12.234841 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe4555-9e75-4182-a772-36c41679e5a3 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:12.319144 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a8284-e2c6-465c-a67e-cc69354c23af None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:12.334545 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:12.347474 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe4555-9e75-4182-a772-36c41679e5a3 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:12.369146 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e0cce-25cf-408a-9d81-9ce40ab67e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:12.372120 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e0cce-25cf-408a-9d81-9ce40ab67e0e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:12.372628 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e0cce-25cf-408a-9d81-9ce40ab67e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:12.373054 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e0cce-25cf-408a-9d81-9ce40ab67e0e None admin] PATH_INFO: `/v3/groups/b411ad5819cf405088a4733141a3b702` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:12.375199 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e0cce-25cf-408a-9d81-9ce40ab67e0e None admin] RBAC: Authorizing `identity:get_group(group_id=b411ad5819cf405088a4733141a3b702)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:12.377398 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e0cce-25cf-408a-9d81-9ce40ab67e0e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:12.380380 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 204/407] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Jul 18 06:06:12 2022] GET /identity/v3/groups/b411ad5819cf405088a4733141a3b702 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:12.381420 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e0cce-25cf-408a-9d81-9ce40ab67e0e None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:12.392038 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4e50e-c4a1-4d48-ad0f-b1916e5d941a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:12.392635 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4e50e-c4a1-4d48-ad0f-b1916e5d941a None admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:12.393149 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4e50e-c4a1-4d48-ad0f-b1916e5d941a None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:12.393531 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4e50e-c4a1-4d48-ad0f-b1916e5d941a None admin] PATH_INFO: `/v3/projects/d34db96dd82a425d90973717021e4c95/groups/b411ad5819cf405088a4733141a3b702/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:12.395821 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4e50e-c4a1-4d48-ad0f-b1916e5d941a None admin] RBAC: Authorizing `identity:create_grant(project_id=d34db96dd82a425d90973717021e4c95, group_id=b411ad5819cf405088a4733141a3b702, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:12.397473 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4e50e-c4a1-4d48-ad0f-b1916e5d941a None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:12.449557 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:12.500925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 204/408] 10.222.0.40 () {58 vars in 1442 bytes} [Mon Jul 18 06:06:12 2022] PUT /identity/v3/projects/d34db96dd82a425d90973717021e4c95/groups/b411ad5819cf405088a4733141a3b702/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:06:12.502130 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4e50e-c4a1-4d48-ad0f-b1916e5d941a None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:12.507029 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e0cce-25cf-408a-9d81-9ce40ab67e0e None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:12.583122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd933b-fd20-4a8f-aa2a-8bad2199e5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:12.584271 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd933b-fd20-4a8f-aa2a-8bad2199e5d0 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:12.586390 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd933b-fd20-4a8f-aa2a-8bad2199e5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:12.586891 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd933b-fd20-4a8f-aa2a-8bad2199e5d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:12.615218 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd933b-fd20-4a8f-aa2a-8bad2199e5d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:12.630203 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-68cd933b-fd20-4a8f-aa2a-8bad2199e5d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:06:12.630203 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:12.630203 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:12.630203 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:12.630203 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:12.630203 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:12.630203 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:12.630203 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:12.630203 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:12.630203 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:12.630203 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:12.630203 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:12.630203 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:12.630203 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:12.630203 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:12.630203 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:12.631365 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:12.631365 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:12.631365 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:12.631365 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:12.631365 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:12.631365 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:12.631365 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:12.631365 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:12.631365 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:12.631365 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:12.631365 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:12.631365 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:12.631365 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:12.631365 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:12.631365 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:12.631365 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:12.632320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:12.632320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:12.632320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:12.632320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:12.632320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:12.632320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:12.632320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:12.632320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:12.632320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:12.632320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:12.632320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:06:12.632320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:12.632320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 205/409] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 18 06:06:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:12.637836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd933b-fd20-4a8f-aa2a-8bad2199e5d0 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:12.686468 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4e50e-c4a1-4d48-ad0f-b1916e5d941a None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:12.727464 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3f28f6-856a-4e97-bdaa-b504809a5793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:12.728628 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f28f6-856a-4e97-bdaa-b504809a5793 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:12.729279 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f28f6-856a-4e97-bdaa-b504809a5793 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:12.729842 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f28f6-856a-4e97-bdaa-b504809a5793 None admin] PATH_INFO: `/v3/projects/d34db96dd82a425d90973717021e4c95` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:12.731241 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3f28f6-856a-4e97-bdaa-b504809a5793 None admin] RBAC: Authorizing `identity:get_project(project_id=d34db96dd82a425d90973717021e4c95)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:12.732911 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3f28f6-856a-4e97-bdaa-b504809a5793 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:12.734891 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 205/410] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:06:12 2022] GET /identity/v3/projects/d34db96dd82a425d90973717021e4c95 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:12.745483 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3f28f6-856a-4e97-bdaa-b504809a5793 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:12.787170 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:12.852001 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd933b-fd20-4a8f-aa2a-8bad2199e5d0 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:12.878093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:12.902814 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3f28f6-856a-4e97-bdaa-b504809a5793 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:12.906585 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc761e75-23b3-40c8-a5bb-4b87b3996297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:12.907283 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc761e75-23b3-40c8-a5bb-4b87b3996297 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:12.907576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc761e75-23b3-40c8-a5bb-4b87b3996297 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:12.907863 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc761e75-23b3-40c8-a5bb-4b87b3996297 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:12.908657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc761e75-23b3-40c8-a5bb-4b87b3996297 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:12.908975 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc761e75-23b3-40c8-a5bb-4b87b3996297 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:12.910409 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc761e75-23b3-40c8-a5bb-4b87b3996297 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:12.931461 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 206/411] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:06:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:06:12.937267 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc761e75-23b3-40c8-a5bb-4b87b3996297 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:12.940604 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de822e-52de-40c0-bd65-a12dff6eb524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:12.941497 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de822e-52de-40c0-bd65-a12dff6eb524 None admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:12.941779 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de822e-52de-40c0-bd65-a12dff6eb524 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:12.942054 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de822e-52de-40c0-bd65-a12dff6eb524 None admin] PATH_INFO: `/v3/projects/d34db96dd82a425d90973717021e4c95/groups/b411ad5819cf405088a4733141a3b702/roles/04868303225144c389068779288f3a42` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:12.944327 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7de822e-52de-40c0-bd65-a12dff6eb524 None admin] RBAC: Authorizing `identity:create_grant(project_id=d34db96dd82a425d90973717021e4c95, group_id=b411ad5819cf405088a4733141a3b702, role_id=04868303225144c389068779288f3a42)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:12.945862 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7de822e-52de-40c0-bd65-a12dff6eb524 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:13.016291 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 206/412] 10.222.0.40 () {58 vars in 1442 bytes} [Mon Jul 18 06:06:12 2022] PUT /identity/v3/projects/d34db96dd82a425d90973717021e4c95/groups/b411ad5819cf405088a4733141a3b702/roles/04868303225144c389068779288f3a42 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:06:13.065864 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc761e75-23b3-40c8-a5bb-4b87b3996297 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:13.112167 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd1e10e-2361-4738-b5f1-5a14d04a0d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:13.113091 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1e10e-2361-4738-b5f1-5a14d04a0d54 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:13.113384 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1e10e-2361-4738-b5f1-5a14d04a0d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:13.113670 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1e10e-2361-4738-b5f1-5a14d04a0d54 None admin] PATH_INFO: `/v3/groups/f3f6c1340437475382e7c75855b0124c` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:13.114899 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd1e10e-2361-4738-b5f1-5a14d04a0d54 None admin] RBAC: Authorizing `identity:get_group(group_id=f3f6c1340437475382e7c75855b0124c)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:13.116364 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd1e10e-2361-4738-b5f1-5a14d04a0d54 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:13.118407 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 207/413] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Jul 18 06:06:12 2022] GET /identity/v3/groups/f3f6c1340437475382e7c75855b0124c => generated 229 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:06:13.123746 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de822e-52de-40c0-bd65-a12dff6eb524 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:13.130141 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf2cc4-9e71-4452-89f7-78239d2ed608 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:13.130482 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf2cc4-9e71-4452-89f7-78239d2ed608 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:13.130829 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf2cc4-9e71-4452-89f7-78239d2ed608 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:13.132393 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 208/414] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:06:13.138496 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0dc115-4419-485f-b21e-f8e7e6a64dce None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:13.139171 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0dc115-4419-485f-b21e-f8e7e6a64dce None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:13.139770 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0dc115-4419-485f-b21e-f8e7e6a64dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:13.179860 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-bd0dc115-4419-485f-b21e-f8e7e6a64dce None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:13.181643 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-bd0dc115-4419-485f-b21e-f8e7e6a64dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:13.266304 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:13.277805 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:13.305787 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 209/415] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 06:06:13.306628 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de822e-52de-40c0-bd65-a12dff6eb524 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:13.333174 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0dc115-4419-485f-b21e-f8e7e6a64dce None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:13.358293 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dddebc2-0dd7-406a-993c-9c52cdc82a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:13.359953 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dddebc2-0dd7-406a-993c-9c52cdc82a42 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:13.360464 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dddebc2-0dd7-406a-993c-9c52cdc82a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:13.361028 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dddebc2-0dd7-406a-993c-9c52cdc82a42 None admin] PATH_INFO: `/v3/projects/3e19cd5e83fe45f5a260277527d6bf8c` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:13.362950 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dddebc2-0dd7-406a-993c-9c52cdc82a42 None admin] RBAC: Authorizing `identity:get_project(project_id=3e19cd5e83fe45f5a260277527d6bf8c)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:13.365103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dddebc2-0dd7-406a-993c-9c52cdc82a42 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:13.367872 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 207/416] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:06:13 2022] GET /identity/v3/projects/3e19cd5e83fe45f5a260277527d6bf8c => generated 344 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:13.376392 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dddebc2-0dd7-406a-993c-9c52cdc82a42 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:13.520743 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0dc115-4419-485f-b21e-f8e7e6a64dce None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:13.593333 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dddebc2-0dd7-406a-993c-9c52cdc82a42 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:13.596221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab9a41f-602b-4fd9-81b7-0e10376ffcd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:13.598225 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9a41f-602b-4fd9-81b7-0e10376ffcd6 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:13.598948 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9a41f-602b-4fd9-81b7-0e10376ffcd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:13.600153 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9a41f-602b-4fd9-81b7-0e10376ffcd6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:13.624894 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab9a41f-602b-4fd9-81b7-0e10376ffcd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:13.640083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-4ab9a41f-602b-4fd9-81b7-0e10376ffcd6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 06:06:13.640083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:13.640083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:13.640083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:13.640083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:13.640083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:13.640083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:13.640083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:13.640083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:13.640083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:13.640083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:13.640083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:13.640083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:13.640083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:13.640083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:13.640083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:13.641073 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:13.641073 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:13.641073 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:13.641073 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:13.641073 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:13.641073 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:13.641073 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:13.641073 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:13.641073 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:13.641073 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:13.641073 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:13.641073 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:13.641073 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:13.641073 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:13.641073 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:13.641073 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:13.641855 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:13.641855 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:13.641855 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:13.641855 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:13.641855 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:13.641855 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:13.641855 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:13.641855 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:13.641855 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:13.641855 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:13.641855 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 06:06:13.641855 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:13.647983 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 210/417] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:06:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:06:13.657263 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab9a41f-602b-4fd9-81b7-0e10376ffcd6 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:13.675164 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e5541c-cd8b-40b6-9f66-acdf0abcc0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:13.676548 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5541c-cd8b-40b6-9f66-acdf0abcc0b0 None admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:13.677091 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5541c-cd8b-40b6-9f66-acdf0abcc0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:13.677563 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5541c-cd8b-40b6-9f66-acdf0abcc0b0 None admin] PATH_INFO: `/v3/projects/3e19cd5e83fe45f5a260277527d6bf8c/groups/f3f6c1340437475382e7c75855b0124c/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:13.683192 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e5541c-cd8b-40b6-9f66-acdf0abcc0b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=3e19cd5e83fe45f5a260277527d6bf8c, group_id=f3f6c1340437475382e7c75855b0124c, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:13.687234 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e5541c-cd8b-40b6-9f66-acdf0abcc0b0 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:13.718279 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:13.780498 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 208/418] 10.222.0.40 () {58 vars in 1442 bytes} [Mon Jul 18 06:06:13 2022] PUT /identity/v3/projects/3e19cd5e83fe45f5a260277527d6bf8c/groups/f3f6c1340437475382e7c75855b0124c/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:06:13.815640 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab9a41f-602b-4fd9-81b7-0e10376ffcd6 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:13.838867 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355b027-4184-40cc-a4ee-884f8bd12409 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:13.839168 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355b027-4184-40cc-a4ee-884f8bd12409 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:13.839453 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355b027-4184-40cc-a4ee-884f8bd12409 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:13.841230 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 209/419] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:06:13.846825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c18cf-57ca-43f2-9df8-92ebb4aa31af None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:13.847169 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c18cf-57ca-43f2-9df8-92ebb4aa31af None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:13.847392 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c18cf-57ca-43f2-9df8-92ebb4aa31af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:13.864810 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:13.872867 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a351bc87-b183-48a6-a6b4-74e85333fc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:13.873654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a351bc87-b183-48a6-a6b4-74e85333fc5f None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:13.874511 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a351bc87-b183-48a6-a6b4-74e85333fc5f None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:13.874511 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a351bc87-b183-48a6-a6b4-74e85333fc5f None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:13.874856 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-807c18cf-57ca-43f2-9df8-92ebb4aa31af None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:13.874966 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a351bc87-b183-48a6-a6b4-74e85333fc5f None admin] RBAC: Adding query filter params (name=member) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:13.875322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a351bc87-b183-48a6-a6b4-74e85333fc5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:13.876810 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a351bc87-b183-48a6-a6b4-74e85333fc5f None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:13.876925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-807c18cf-57ca-43f2-9df8-92ebb4aa31af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:13.895816 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 211/420] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 18 06:06:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:13.902051 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a351bc87-b183-48a6-a6b4-74e85333fc5f None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:14.041941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 210/421] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 06:06:14.085877 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:14.090570 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c18cf-57ca-43f2-9df8-92ebb4aa31af None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:14.091015 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a351bc87-b183-48a6-a6b4-74e85333fc5f None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:14.147049 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115ddd2-fc5f-45ee-888f-b529730579df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:14.147771 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115ddd2-fc5f-45ee-888f-b529730579df None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:14.148054 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115ddd2-fc5f-45ee-888f-b529730579df None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:14.148377 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115ddd2-fc5f-45ee-888f-b529730579df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:14.280322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115ddd2-fc5f-45ee-888f-b529730579df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:14.281879 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115ddd2-fc5f-45ee-888f-b529730579df None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:14.286208 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c18cf-57ca-43f2-9df8-92ebb4aa31af None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:14.384828 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732ae27-5dd9-4f4e-8c16-819e8633c98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:14.386677 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732ae27-5dd9-4f4e-8c16-819e8633c98e None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:14.387287 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732ae27-5dd9-4f4e-8c16-819e8633c98e None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:14.387788 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732ae27-5dd9-4f4e-8c16-819e8633c98e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:14.395273 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:14.408333 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:14.410588 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0732ae27-5dd9-4f4e-8c16-819e8633c98e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:14.413835 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 212/422] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:06:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 06:06:14.422558 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-0732ae27-5dd9-4f4e-8c16-819e8633c98e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 06:06:14.422558 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:14.422558 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:14.422558 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:14.422558 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:14.422558 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:14.422558 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:14.422558 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:14.422558 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:14.422558 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:14.422558 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:14.422558 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:14.422558 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:14.422558 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:14.422558 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:14.422558 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:14.423610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:14.423610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:14.423610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:14.423610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:14.423610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:14.423610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:14.423610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:14.423610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:14.423610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:14.423610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:14.423610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:14.423610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:14.423610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:14.423610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:14.423610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:14.423610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:14.424436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:14.424436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:14.424436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:14.424436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:14.424436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:14.424436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:14.424436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:14.424436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:14.424436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:14.424436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:14.424436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 06:06:14.424436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:14.425077 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115ddd2-fc5f-45ee-888f-b529730579df None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:14.440612 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 211/423] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 18 06:06:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:14.441748 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732ae27-5dd9-4f4e-8c16-819e8633c98e None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:14.570172 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115ddd2-fc5f-45ee-888f-b529730579df None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:14.594085 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732ae27-5dd9-4f4e-8c16-819e8633c98e None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:14.638636 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba6318-36ba-443f-a951-5a34e32867d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:14.639337 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba6318-36ba-443f-a951-5a34e32867d6 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:14.639783 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba6318-36ba-443f-a951-5a34e32867d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:14.640129 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba6318-36ba-443f-a951-5a34e32867d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:14.640576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc7daac-1bbd-4918-bbec-b507cb8f7aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:14.641057 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba6318-36ba-443f-a951-5a34e32867d6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:14.641586 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba6318-36ba-443f-a951-5a34e32867d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:14.641938 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7daac-1bbd-4918-bbec-b507cb8f7aac None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:14.642372 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7daac-1bbd-4918-bbec-b507cb8f7aac None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:14.642798 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7daac-1bbd-4918-bbec-b507cb8f7aac None admin] PATH_INFO: `/v3/projects/4620d11733d2401eb2122d406221cf0d` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:14.643225 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba6318-36ba-443f-a951-5a34e32867d6 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:14.649184 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc7daac-1bbd-4918-bbec-b507cb8f7aac None admin] RBAC: Authorizing `identity:get_project(project_id=4620d11733d2401eb2122d406221cf0d)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:14.650844 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc7daac-1bbd-4918-bbec-b507cb8f7aac None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:14.653109 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 213/425] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:06:14 2022] GET /identity/v3/projects/4620d11733d2401eb2122d406221cf0d => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:14.653224 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 212/425] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:06:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:14.658696 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc7daac-1bbd-4918-bbec-b507cb8f7aac None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:14.658856 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba6318-36ba-443f-a951-5a34e32867d6 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:14.723930 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:14.773059 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba6318-36ba-443f-a951-5a34e32867d6 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:14.809793 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:14.815024 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e2bf9-455b-407b-b6e6-50980705dcfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:14.815894 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e2bf9-455b-407b-b6e6-50980705dcfd None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:14.816178 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e2bf9-455b-407b-b6e6-50980705dcfd None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:14.816472 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e2bf9-455b-407b-b6e6-50980705dcfd None admin] PATH_INFO: `/v3/groups/b411ad5819cf405088a4733141a3b702` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:14.817646 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193e2bf9-455b-407b-b6e6-50980705dcfd None admin] RBAC: Authorizing `identity:get_group(group_id=b411ad5819cf405088a4733141a3b702)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:14.819084 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193e2bf9-455b-407b-b6e6-50980705dcfd None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:14.821087 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 213/426] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Jul 18 06:06:14 2022] GET /identity/v3/groups/b411ad5819cf405088a4733141a3b702 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:06:14.827973 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e2bf9-455b-407b-b6e6-50980705dcfd None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:14.878025 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc7daac-1bbd-4918-bbec-b507cb8f7aac None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:14.941249 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e2bf9-455b-407b-b6e6-50980705dcfd None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:14.957032 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2891b5fa-b513-46de-8526-7b699e867b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:14.958111 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2891b5fa-b513-46de-8526-7b699e867b13 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:14.958595 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2891b5fa-b513-46de-8526-7b699e867b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:14.959070 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2891b5fa-b513-46de-8526-7b699e867b13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:14.968966 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:14.980658 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c4e2c-20a3-4284-b0cd-d58f7b6aa28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:14.982553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c4e2c-20a3-4284-b0cd-d58f7b6aa28e None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:14.982553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c4e2c-20a3-4284-b0cd-d58f7b6aa28e None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:14.982553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c4e2c-20a3-4284-b0cd-d58f7b6aa28e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:14.982955 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776c4e2c-20a3-4284-b0cd-d58f7b6aa28e None admin] RBAC: Adding query filter params (group.id=b411ad5819cf405088a4733141a3b702, role.id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20, scope.project.id=4620d11733d2401eb2122d406221cf0d) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:14.983325 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776c4e2c-20a3-4284-b0cd-d58f7b6aa28e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:14.984901 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776c4e2c-20a3-4284-b0cd-d58f7b6aa28e None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:14.991723 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 214/427] 10.222.0.40 () {56 vars in 1376 bytes} [Mon Jul 18 06:06:14 2022] GET /identity/v3/role_assignments?group.id=b411ad5819cf405088a4733141a3b702&scope.project.id=4620d11733d2401eb2122d406221cf0d&role.id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 611 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:06:15.094919 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2891b5fa-b513-46de-8526-7b699e867b13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:15.096441 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2891b5fa-b513-46de-8526-7b699e867b13 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:15.188179 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:15.227372 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 214/428] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:06:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 06:06:15.233390 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c4e2c-20a3-4284-b0cd-d58f7b6aa28e None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:15.302274 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea66185-0165-4d87-8fc9-ece78c4ca5f3 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:15.302913 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea66185-0165-4d87-8fc9-ece78c4ca5f3 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:15.303201 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea66185-0165-4d87-8fc9-ece78c4ca5f3 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:15.305273 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 215/429] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:06:15.312427 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d80c8b-9c87-4ed6-9c5c-08fe656a9304 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:15.312732 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d80c8b-9c87-4ed6-9c5c-08fe656a9304 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:15.313014 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d80c8b-9c87-4ed6-9c5c-08fe656a9304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:15.335431 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-82d80c8b-9c87-4ed6-9c5c-08fe656a9304 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:15.336764 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-82d80c8b-9c87-4ed6-9c5c-08fe656a9304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:15.362032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c4e2c-20a3-4284-b0cd-d58f7b6aa28e None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:15.411846 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26957c5f-f682-4db4-807e-37734e43a3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:15.413110 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26957c5f-f682-4db4-807e-37734e43a3d0 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:15.413452 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26957c5f-f682-4db4-807e-37734e43a3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:15.413733 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26957c5f-f682-4db4-807e-37734e43a3d0 None admin] PATH_INFO: `/v3/projects/4620d11733d2401eb2122d406221cf0d` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:15.415245 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26957c5f-f682-4db4-807e-37734e43a3d0 None admin] RBAC: Authorizing `identity:get_project(project_id=4620d11733d2401eb2122d406221cf0d)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:15.417415 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26957c5f-f682-4db4-807e-37734e43a3d0 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:15.419914 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 215/430] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:06:15 2022] GET /identity/v3/projects/4620d11733d2401eb2122d406221cf0d => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:15.425146 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26957c5f-f682-4db4-807e-37734e43a3d0 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:15.453892 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 216/431] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:06:15.478562 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d80c8b-9c87-4ed6-9c5c-08fe656a9304 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:15.499406 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:15.554379 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26957c5f-f682-4db4-807e-37734e43a3d0 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:15.594694 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc558a0-c6a0-44ab-acdc-960491264078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:15.596322 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc558a0-c6a0-44ab-acdc-960491264078 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:15.597557 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc558a0-c6a0-44ab-acdc-960491264078 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:15.598036 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc558a0-c6a0-44ab-acdc-960491264078 None admin] PATH_INFO: `/v3/groups/b411ad5819cf405088a4733141a3b702` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:15.599462 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc558a0-c6a0-44ab-acdc-960491264078 None admin] RBAC: Authorizing `identity:get_group(group_id=b411ad5819cf405088a4733141a3b702)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:15.601143 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc558a0-c6a0-44ab-acdc-960491264078 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:15.603272 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 216/432] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Jul 18 06:06:15 2022] GET /identity/v3/groups/b411ad5819cf405088a4733141a3b702 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:15.604788 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a302b-59db-488f-b1d4-b7c1f7943d49 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:15.605287 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a302b-59db-488f-b1d4-b7c1f7943d49 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:15.605761 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a302b-59db-488f-b1d4-b7c1f7943d49 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:15.610576 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 217/433] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:15.610991 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:15.612810 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a302b-59db-488f-b1d4-b7c1f7943d49 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:15.618585 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d80c8b-9c87-4ed6-9c5c-08fe656a9304 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:15.660783 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db77e1a-1785-4688-b694-cc18ec3a6cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:15.661715 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db77e1a-1785-4688-b694-cc18ec3a6cfe None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:15.661999 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db77e1a-1785-4688-b694-cc18ec3a6cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:15.662530 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db77e1a-1785-4688-b694-cc18ec3a6cfe None admin] PATH_INFO: `/v3/roles/member` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:15.673780 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db77e1a-1785-4688-b694-cc18ec3a6cfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:15.683040 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-1db77e1a-1785-4688-b694-cc18ec3a6cfe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 06:06:15.683040 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:15.683040 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:15.683040 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:15.683040 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:15.683040 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:15.683040 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:15.683040 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:15.683040 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:15.683040 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:15.683040 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:15.683040 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:15.683040 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:15.683040 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:15.683040 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:15.683040 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:15.683992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:15.683992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:15.683992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:15.683992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:15.683992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:15.683992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:15.683992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:15.683992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:15.683992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:15.683992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:15.683992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:15.683992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:15.683992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:15.683992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:15.683992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:15.683992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:15.685064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:15.685064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:15.685064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:15.685064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:15.685064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:15.685064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:15.685064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:15.685064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:15.685064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:15.685064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:15.685064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 06:06:15.685064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:15.685064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 217/434] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:06:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:06:15.685900 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68152397-e779-4734-9ce7-18fb33b61798 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:15.686171 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68152397-e779-4734-9ce7-18fb33b61798 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:15.686414 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68152397-e779-4734-9ce7-18fb33b61798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:15.710604 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-68152397-e779-4734-9ce7-18fb33b61798 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:15.711939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-68152397-e779-4734-9ce7-18fb33b61798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:15.762008 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a302b-59db-488f-b1d4-b7c1f7943d49 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:15.787021 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 218/435] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:06:15.788130 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68152397-e779-4734-9ce7-18fb33b61798 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:15.795246 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:15.805548 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5020a6e8-f7ac-4336-b118-23cdf0abab62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:15.806611 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020a6e8-f7ac-4336-b118-23cdf0abab62 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:15.807632 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020a6e8-f7ac-4336-b118-23cdf0abab62 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:15.808566 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020a6e8-f7ac-4336-b118-23cdf0abab62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:15.809768 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5020a6e8-f7ac-4336-b118-23cdf0abab62 None admin] RBAC: Adding query filter params (group.id=b411ad5819cf405088a4733141a3b702, role.id=04868303225144c389068779288f3a42, scope.project.id=4620d11733d2401eb2122d406221cf0d) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:15.810282 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5020a6e8-f7ac-4336-b118-23cdf0abab62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:15.812002 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5020a6e8-f7ac-4336-b118-23cdf0abab62 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:15.818669 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 218/436] 10.222.0.40 () {56 vars in 1376 bytes} [Mon Jul 18 06:06:15 2022] GET /identity/v3/role_assignments?group.id=b411ad5819cf405088a4733141a3b702&scope.project.id=4620d11733d2401eb2122d406221cf0d&role.id=04868303225144c389068779288f3a42 => generated 611 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:06:15.819785 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5020a6e8-f7ac-4336-b118-23cdf0abab62 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:15.922936 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:15.945339 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68152397-e779-4734-9ce7-18fb33b61798 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:15.955668 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5020a6e8-f7ac-4336-b118-23cdf0abab62 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:15.988305 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f5bd27-572c-4dad-9e13-bd8315671185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:15.989438 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f5bd27-572c-4dad-9e13-bd8315671185 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:15.989965 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f5bd27-572c-4dad-9e13-bd8315671185 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:15.991108 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f5bd27-572c-4dad-9e13-bd8315671185 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:15.991895 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f5bd27-572c-4dad-9e13-bd8315671185 None admin] RBAC: Adding query filter params (name=member) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:15.992202 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f5bd27-572c-4dad-9e13-bd8315671185 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:15.993649 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f5bd27-572c-4dad-9e13-bd8315671185 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:16.005650 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 219/437] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 18 06:06:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:16.007148 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e4db1-2d8d-4367-bddc-f4422710bade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:16.007980 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e4db1-2d8d-4367-bddc-f4422710bade None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:16.008268 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e4db1-2d8d-4367-bddc-f4422710bade None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:16.008585 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e4db1-2d8d-4367-bddc-f4422710bade None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:16.011564 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f5bd27-572c-4dad-9e13-bd8315671185 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:16.023265 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e4db1-2d8d-4367-bddc-f4422710bade None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:16.034367 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-cd2e4db1-2d8d-4367-bddc-f4422710bade None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 06:06:16.034367 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:16.034367 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:16.034367 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:16.034367 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:16.034367 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:16.034367 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:16.034367 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:16.034367 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:16.034367 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:16.034367 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:16.034367 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:16.034367 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:16.034367 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:16.034367 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:16.034367 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:16.035342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:16.035342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:16.035342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:16.035342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:16.035342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:16.035342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:16.035342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:16.035342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:16.035342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:16.035342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:16.035342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:16.035342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:16.035342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:16.035342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:16.035342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:16.035342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:16.036186 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:16.036186 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:16.036186 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:16.036186 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:16.036186 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:16.036186 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:16.036186 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:16.036186 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:16.036186 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:16.036186 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:16.036186 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 06:06:16.036186 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:16.037599 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 219/438] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 18 06:06:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:16.045446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e4db1-2d8d-4367-bddc-f4422710bade None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:16.108348 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:16.145491 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f5bd27-572c-4dad-9e13-bd8315671185 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:16.173077 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e4db1-2d8d-4367-bddc-f4422710bade None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:16.191001 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a39ff3c-de28-4eee-b7c3-4bb08c503f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:16.191695 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a39ff3c-de28-4eee-b7c3-4bb08c503f89 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:16.191991 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a39ff3c-de28-4eee-b7c3-4bb08c503f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:16.192275 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a39ff3c-de28-4eee-b7c3-4bb08c503f89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:16.212535 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:16.217683 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14455b-859b-4116-a1f1-725f9de346e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:16.218571 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14455b-859b-4116-a1f1-725f9de346e3 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:16.218988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14455b-859b-4116-a1f1-725f9de346e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:16.219406 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14455b-859b-4116-a1f1-725f9de346e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:16.223958 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14455b-859b-4116-a1f1-725f9de346e3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:16.223958 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14455b-859b-4116-a1f1-725f9de346e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:16.223958 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14455b-859b-4116-a1f1-725f9de346e3 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:16.232187 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 220/439] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:06:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:06:16.237941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14455b-859b-4116-a1f1-725f9de346e3 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:16.288763 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a39ff3c-de28-4eee-b7c3-4bb08c503f89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:16.290166 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a39ff3c-de28-4eee-b7c3-4bb08c503f89 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:16.382285 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14455b-859b-4116-a1f1-725f9de346e3 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:16.393564 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:16.408312 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 220/440] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:06:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 06:06:16.409764 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0ba4d-2b3b-41a8-b574-c28ef47cfbe8 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:16.410064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0ba4d-2b3b-41a8-b574-c28ef47cfbe8 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:16.410351 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0ba4d-2b3b-41a8-b574-c28ef47cfbe8 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:16.411652 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 221/441] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:16.414375 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f0ba4d-2b3b-41a8-b574-c28ef47cfbe8 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:16.418894 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8bdc6-c3f1-4685-8754-56b0025f201f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:16.419553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8bdc6-c3f1-4685-8754-56b0025f201f None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:16.419836 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8bdc6-c3f1-4685-8754-56b0025f201f None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:16.420120 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8bdc6-c3f1-4685-8754-56b0025f201f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:16.466514 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:16.489965 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff8bdc6-c3f1-4685-8754-56b0025f201f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:16.491452 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff8bdc6-c3f1-4685-8754-56b0025f201f None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:16.524706 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f0ba4d-2b3b-41a8-b574-c28ef47cfbe8 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:16.564704 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0275eebf-58f6-46a0-a64a-b09f4f4fdc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:16.565489 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275eebf-58f6-46a0-a64a-b09f4f4fdc45 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:16.565782 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275eebf-58f6-46a0-a64a-b09f4f4fdc45 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:16.566062 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275eebf-58f6-46a0-a64a-b09f4f4fdc45 None admin] PATH_INFO: `/v3/projects/d34db96dd82a425d90973717021e4c95` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:16.567266 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0275eebf-58f6-46a0-a64a-b09f4f4fdc45 None admin] RBAC: Authorizing `identity:get_project(project_id=d34db96dd82a425d90973717021e4c95)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:16.568771 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0275eebf-58f6-46a0-a64a-b09f4f4fdc45 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:16.570484 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 222/442] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:06:16 2022] GET /identity/v3/projects/d34db96dd82a425d90973717021e4c95 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:16.570484 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f2621-aee0-4f18-9224-99918cf7c4d6 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:16.570484 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f2621-aee0-4f18-9224-99918cf7c4d6 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:16.570484 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f2621-aee0-4f18-9224-99918cf7c4d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:16.590477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-050f2621-aee0-4f18-9224-99918cf7c4d6 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:16.591719 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-050f2621-aee0-4f18-9224-99918cf7c4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:16.599049 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 221/443] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:06:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 06:06:16.600051 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8bdc6-c3f1-4685-8754-56b0025f201f None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:16.612254 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:16.663851 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 223/444] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:06:16.669349 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050f2621-aee0-4f18-9224-99918cf7c4d6 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:16.710226 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8bdc6-c3f1-4685-8754-56b0025f201f None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:16.721310 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:16.746486 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18beb6c-c700-4174-8b20-3db65df63668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:16.747343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18beb6c-c700-4174-8b20-3db65df63668 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:16.747627 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18beb6c-c700-4174-8b20-3db65df63668 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:16.747903 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18beb6c-c700-4174-8b20-3db65df63668 None admin] PATH_INFO: `/v3/groups/b411ad5819cf405088a4733141a3b702` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:16.749120 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18beb6c-c700-4174-8b20-3db65df63668 None admin] RBAC: Authorizing `identity:get_group(group_id=b411ad5819cf405088a4733141a3b702)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:16.750579 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18beb6c-c700-4174-8b20-3db65df63668 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:16.752250 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 222/445] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Jul 18 06:06:16 2022] GET /identity/v3/groups/b411ad5819cf405088a4733141a3b702 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:16.753056 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18beb6c-c700-4174-8b20-3db65df63668 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:16.776594 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050f2621-aee0-4f18-9224-99918cf7c4d6 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:16.815493 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225153cd-df31-41f9-ae21-f75a5a060c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:16.815493 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225153cd-df31-41f9-ae21-f75a5a060c17 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:16.815493 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225153cd-df31-41f9-ae21-f75a5a060c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:16.815493 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225153cd-df31-41f9-ae21-f75a5a060c17 None admin] PATH_INFO: `/v3/projects/d34db96dd82a425d90973717021e4c95` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:16.816580 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225153cd-df31-41f9-ae21-f75a5a060c17 None admin] RBAC: Authorizing `identity:get_project(project_id=d34db96dd82a425d90973717021e4c95)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:16.818041 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225153cd-df31-41f9-ae21-f75a5a060c17 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:16.819777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 224/446] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:06:16 2022] GET /identity/v3/projects/d34db96dd82a425d90973717021e4c95 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:16.820579 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225153cd-df31-41f9-ae21-f75a5a060c17 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:16.846200 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:16.868237 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18beb6c-c700-4174-8b20-3db65df63668 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:16.903887 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc7bbd-0cb7-452f-b419-0390cd4bf3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:16.904708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc7bbd-0cb7-452f-b419-0390cd4bf3ba None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:16.904993 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc7bbd-0cb7-452f-b419-0390cd4bf3ba None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:16.905267 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc7bbd-0cb7-452f-b419-0390cd4bf3ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:16.915965 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fc7bbd-0cb7-452f-b419-0390cd4bf3ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:16.924655 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-96fc7bbd-0cb7-452f-b419-0390cd4bf3ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:06:16.924655 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:16.924655 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:16.924655 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:16.924655 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:16.924655 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:16.924655 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:16.924655 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:16.924655 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:16.924655 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:16.924655 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:16.924655 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:16.924655 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:16.924655 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:16.924655 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:16.924655 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:16.925772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:16.925772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:16.925772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:16.925772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:16.925772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:16.925772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:16.925772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:16.925772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:16.925772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:16.925772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:16.925772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:16.925772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:16.925772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:16.925772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:16.925772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:16.925772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:16.926619 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:16.926619 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:16.926619 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:16.926619 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:16.926619 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:16.926619 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:16.926619 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:16.926619 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:16.926619 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:16.926619 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:16.926619 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:06:16.926619 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:16.927234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225153cd-df31-41f9-ae21-f75a5a060c17 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:16.927312 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 223/447] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 18 06:06:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:16.927312 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc7bbd-0cb7-452f-b419-0390cd4bf3ba None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:16.944929 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:16.967500 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acfc880-854e-4abb-b80a-1240095077f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:16.968302 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfc880-854e-4abb-b80a-1240095077f3 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:16.968616 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfc880-854e-4abb-b80a-1240095077f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:16.968897 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfc880-854e-4abb-b80a-1240095077f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:16.969744 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acfc880-854e-4abb-b80a-1240095077f3 None admin] RBAC: Adding query filter params (group.id=b411ad5819cf405088a4733141a3b702, role.id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20, scope.project.id=d34db96dd82a425d90973717021e4c95) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:16.970053 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acfc880-854e-4abb-b80a-1240095077f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:16.971589 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acfc880-854e-4abb-b80a-1240095077f3 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:16.978196 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 225/448] 10.222.0.40 () {56 vars in 1376 bytes} [Mon Jul 18 06:06:16 2022] GET /identity/v3/role_assignments?group.id=b411ad5819cf405088a4733141a3b702&scope.project.id=d34db96dd82a425d90973717021e4c95&role.id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:16.979563 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acfc880-854e-4abb-b80a-1240095077f3 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:17.040052 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc7bbd-0cb7-452f-b419-0390cd4bf3ba None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:17.087191 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e6f643-06c1-4736-8c5e-3af3553f3051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:17.089468 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6f643-06c1-4736-8c5e-3af3553f3051 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:17.089981 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6f643-06c1-4736-8c5e-3af3553f3051 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:17.090480 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6f643-06c1-4736-8c5e-3af3553f3051 None admin] PATH_INFO: `/v3/groups/b411ad5819cf405088a4733141a3b702` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:17.092880 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e6f643-06c1-4736-8c5e-3af3553f3051 None admin] RBAC: Authorizing `identity:get_group(group_id=b411ad5819cf405088a4733141a3b702)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:17.096006 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e6f643-06c1-4736-8c5e-3af3553f3051 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:17.099573 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 224/449] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Jul 18 06:06:16 2022] GET /identity/v3/groups/b411ad5819cf405088a4733141a3b702 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:17.105731 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e6f643-06c1-4736-8c5e-3af3553f3051 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:17.107380 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acfc880-854e-4abb-b80a-1240095077f3 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:17.117701 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:17.154166 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a846f-1c9b-4a1a-acec-133303d969e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:17.155180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a846f-1c9b-4a1a-acec-133303d969e9 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:17.155475 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a846f-1c9b-4a1a-acec-133303d969e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:17.155759 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a846f-1c9b-4a1a-acec-133303d969e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:17.157616 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35a846f-1c9b-4a1a-acec-133303d969e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:17.157947 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35a846f-1c9b-4a1a-acec-133303d969e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:17.159370 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35a846f-1c9b-4a1a-acec-133303d969e9 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:17.171945 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 226/450] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:06:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:17.178618 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a846f-1c9b-4a1a-acec-133303d969e9 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:17.235935 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:17.243215 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e6f643-06c1-4736-8c5e-3af3553f3051 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:17.293289 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7750754c-8a3d-4c48-b5e0-a77a4629cb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:17.293958 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750754c-8a3d-4c48-b5e0-a77a4629cb9a None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:17.294242 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750754c-8a3d-4c48-b5e0-a77a4629cb9a None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:17.294524 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750754c-8a3d-4c48-b5e0-a77a4629cb9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:17.295376 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7750754c-8a3d-4c48-b5e0-a77a4629cb9a None admin] RBAC: Adding query filter params (group.id=b411ad5819cf405088a4733141a3b702, role.id=04868303225144c389068779288f3a42, scope.project.id=d34db96dd82a425d90973717021e4c95) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:17.295690 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7750754c-8a3d-4c48-b5e0-a77a4629cb9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:17.299976 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7750754c-8a3d-4c48-b5e0-a77a4629cb9a None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:17.305079 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 225/451] 10.222.0.40 () {56 vars in 1376 bytes} [Mon Jul 18 06:06:17 2022] GET /identity/v3/role_assignments?group.id=b411ad5819cf405088a4733141a3b702&scope.project.id=d34db96dd82a425d90973717021e4c95&role.id=04868303225144c389068779288f3a42 => generated 611 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:17.317535 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a846f-1c9b-4a1a-acec-133303d969e9 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:17.358500 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf21c6-178a-4a56-8c56-1985694c74f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:17.359419 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf21c6-178a-4a56-8c56-1985694c74f5 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:17.359896 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf21c6-178a-4a56-8c56-1985694c74f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:17.360185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf21c6-178a-4a56-8c56-1985694c74f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:17.442675 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adf21c6-178a-4a56-8c56-1985694c74f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:17.444281 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adf21c6-178a-4a56-8c56-1985694c74f5 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:17.491829 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:17.563949 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 227/452] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:06:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 06:06:17.569943 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7750754c-8a3d-4c48-b5e0-a77a4629cb9a None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:17.645856 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:17.677993 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7750754c-8a3d-4c48-b5e0-a77a4629cb9a None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:17.712287 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244c21f-3342-408f-9c03-526f856859ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:17.713755 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244c21f-3342-408f-9c03-526f856859ce None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:17.714210 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244c21f-3342-408f-9c03-526f856859ce None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:17.714556 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244c21f-3342-408f-9c03-526f856859ce None admin] PATH_INFO: `/v3/projects/3e19cd5e83fe45f5a260277527d6bf8c` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:17.715785 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c244c21f-3342-408f-9c03-526f856859ce None admin] RBAC: Authorizing `identity:get_project(project_id=3e19cd5e83fe45f5a260277527d6bf8c)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:17.717247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c244c21f-3342-408f-9c03-526f856859ce None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:17.718934 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 226/453] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:06:17 2022] GET /identity/v3/projects/3e19cd5e83fe45f5a260277527d6bf8c => generated 344 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:17.724615 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf21c6-178a-4a56-8c56-1985694c74f5 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:17.824400 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf21c6-178a-4a56-8c56-1985694c74f5 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:17.858773 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e7574-35a4-4faf-a963-772c62fd5a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:17.859666 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e7574-35a4-4faf-a963-772c62fd5a7b None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:17.859954 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e7574-35a4-4faf-a963-772c62fd5a7b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:17.860235 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e7574-35a4-4faf-a963-772c62fd5a7b None admin] PATH_INFO: `/v3/groups/f3f6c1340437475382e7c75855b0124c` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:17.861434 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7e7574-35a4-4faf-a963-772c62fd5a7b None admin] RBAC: Authorizing `identity:get_group(group_id=f3f6c1340437475382e7c75855b0124c)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:17.862899 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7e7574-35a4-4faf-a963-772c62fd5a7b None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:17.864557 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 228/454] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Jul 18 06:06:17 2022] GET /identity/v3/groups/f3f6c1340437475382e7c75855b0124c => generated 229 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:17.871237 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244c21f-3342-408f-9c03-526f856859ce None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:17.968299 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244c21f-3342-408f-9c03-526f856859ce None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:18.001437 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a25ff5-e08c-4fa5-8847-902c3f1e5f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:18.002085 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a25ff5-e08c-4fa5-8847-902c3f1e5f69 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:18.002372 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a25ff5-e08c-4fa5-8847-902c3f1e5f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:18.002657 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a25ff5-e08c-4fa5-8847-902c3f1e5f69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:18.003492 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a25ff5-e08c-4fa5-8847-902c3f1e5f69 None admin] RBAC: Adding query filter params (group.id=f3f6c1340437475382e7c75855b0124c, role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc, scope.project.id=3e19cd5e83fe45f5a260277527d6bf8c) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:18.003808 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a25ff5-e08c-4fa5-8847-902c3f1e5f69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:18.005265 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a25ff5-e08c-4fa5-8847-902c3f1e5f69 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:18.011203 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 227/455] 10.222.0.40 () {56 vars in 1376 bytes} [Mon Jul 18 06:06:17 2022] GET /identity/v3/role_assignments?group.id=f3f6c1340437475382e7c75855b0124c&scope.project.id=3e19cd5e83fe45f5a260277527d6bf8c&role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:20.498444 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be8fae-7b34-408a-bbe7-76e70d7c3a9d None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:20.498995 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be8fae-7b34-408a-bbe7-76e70d7c3a9d None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:20.498995 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be8fae-7b34-408a-bbe7-76e70d7c3a9d None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:20.500687 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 229/456] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:20.506186 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d56f3b-b583-4bf0-abad-508984d15dbd None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:20.506642 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d56f3b-b583-4bf0-abad-508984d15dbd None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:20.506740 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d56f3b-b583-4bf0-abad-508984d15dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:20.512009 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:20.529517 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-61d56f3b-b583-4bf0-abad-508984d15dbd None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:20.530722 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-61d56f3b-b583-4bf0-abad-508984d15dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:20.629253 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 228/457] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 06:06:20.687377 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be8fae-7b34-408a-bbe7-76e70d7c3a9d None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:20.702673 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:20.774184 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea1453-b202-40ad-8269-7bc51c562d29 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:20.774807 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea1453-b202-40ad-8269-7bc51c562d29 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:20.775294 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea1453-b202-40ad-8269-7bc51c562d29 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:20.779600 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 229/458] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:06:20.785095 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f0e92f-7472-4ae1-89ab-523f63c6aa67 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:20.785919 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f0e92f-7472-4ae1-89ab-523f63c6aa67 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:20.786370 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f0e92f-7472-4ae1-89ab-523f63c6aa67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:20.809432 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-68f0e92f-7472-4ae1-89ab-523f63c6aa67 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:20.812186 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-68f0e92f-7472-4ae1-89ab-523f63c6aa67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:20.881168 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be8fae-7b34-408a-bbe7-76e70d7c3a9d None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:20.908751 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 230/459] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 06:06:20.929053 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f0e92f-7472-4ae1-89ab-523f63c6aa67 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:20.967208 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d98a8-b7d5-4bcb-9c23-ac0efc34c641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:20.968146 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d98a8-b7d5-4bcb-9c23-ac0efc34c641 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:20.968453 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d98a8-b7d5-4bcb-9c23-ac0efc34c641 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:20.968740 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d98a8-b7d5-4bcb-9c23-ac0efc34c641 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:20.988905 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d98a8-b7d5-4bcb-9c23-ac0efc34c641 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:20.990740 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d98a8-b7d5-4bcb-9c23-ac0efc34c641 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:21.012793 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:21.019467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-871d98a8-b7d5-4bcb-9c23-ac0efc34c641 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:21.019467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:21.019467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:21.019467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:21.019467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:21.019467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:21.019467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:21.019467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:21.019467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:21.019467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:21.019467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:21.019467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:21.019467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:21.019467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:21.019467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:21.020392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:21.020392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:21.020392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:21.020392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:21.020392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:21.020392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:21.020392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:21.020392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:21.020392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:21.020392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:21.020392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:21.020392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:21.020392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:21.020392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:21.020392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:21.020392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:21.020392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:21.021291 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:21.021291 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:21.021291 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:21.021291 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:21.021291 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 230/460] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:21.022510 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c7bc0-6d9a-4806-9033-46d1460be4fa None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:21.022805 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c7bc0-6d9a-4806-9033-46d1460be4fa None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:21.023083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c7bc0-6d9a-4806-9033-46d1460be4fa None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:21.029531 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 231/461] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:21.031076 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6301474-9e13-4795-a174-e1aaa246fedf None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:21.031564 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6301474-9e13-4795-a174-e1aaa246fedf None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:21.031993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6301474-9e13-4795-a174-e1aaa246fedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:21.051861 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-c6301474-9e13-4795-a174-e1aaa246fedf None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:21.053426 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c6301474-9e13-4795-a174-e1aaa246fedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:21.064380 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f0e92f-7472-4ae1-89ab-523f63c6aa67 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:21.111609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1732bba4-6f0c-49e4-9e9a-ccdfc7ea4824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:21.113201 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732bba4-6f0c-49e4-9e9a-ccdfc7ea4824 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:21.113925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732bba4-6f0c-49e4-9e9a-ccdfc7ea4824 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:21.114366 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732bba4-6f0c-49e4-9e9a-ccdfc7ea4824 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:21.121932 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:21.129000 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1732bba4-6f0c-49e4-9e9a-ccdfc7ea4824 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:21.132589 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1732bba4-6f0c-49e4-9e9a-ccdfc7ea4824 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:21.144723 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 232/462] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:06:21.145818 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6301474-9e13-4795-a174-e1aaa246fedf None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:21.153455 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-1732bba4-6f0c-49e4-9e9a-ccdfc7ea4824 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:21.153455 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:21.153455 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:21.153455 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:21.153455 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:21.153455 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:21.153455 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:21.153455 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:21.153455 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:21.153455 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:21.153455 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:21.153455 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:21.153455 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:21.153455 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:21.153455 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:21.154563 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:21.154563 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:21.154563 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:21.154563 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:21.154563 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:21.154563 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:21.154563 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:21.154563 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:21.154563 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:21.154563 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:21.154563 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:21.154563 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:21.154563 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:21.154563 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:21.154563 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:21.154563 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:21.154563 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:21.155676 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:21.155676 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:21.155676 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:21.155676 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:21.160969 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 231/463] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:06:21.160969 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc02c8-315f-4f21-911e-b14ac9b28029 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:21.160969 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc02c8-315f-4f21-911e-b14ac9b28029 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:21.160969 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc02c8-315f-4f21-911e-b14ac9b28029 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:21.164454 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 232/464] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:21.165661 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc00ef-5a0a-4d24-8214-c0571e5e6c9f None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:21.166008 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc00ef-5a0a-4d24-8214-c0571e5e6c9f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:21.167335 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc00ef-5a0a-4d24-8214-c0571e5e6c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:21.185614 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-ebdc00ef-5a0a-4d24-8214-c0571e5e6c9f None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:21.186834 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ebdc00ef-5a0a-4d24-8214-c0571e5e6c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:21.288057 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 233/465] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:06:21.289282 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc00ef-5a0a-4d24-8214-c0571e5e6c9f None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:21.298588 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6301474-9e13-4795-a174-e1aaa246fedf None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:21.340839 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01439da-568b-4442-b7f2-249ab315740d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:21.341523 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01439da-568b-4442-b7f2-249ab315740d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:21.341812 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01439da-568b-4442-b7f2-249ab315740d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:21.342094 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01439da-568b-4442-b7f2-249ab315740d None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:21.342830 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01439da-568b-4442-b7f2-249ab315740d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:21.343139 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01439da-568b-4442-b7f2-249ab315740d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:21.345308 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01439da-568b-4442-b7f2-249ab315740d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:21.347903 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:21.359548 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 233/466] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:21.360400 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01439da-568b-4442-b7f2-249ab315740d None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:21.420148 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:21.421096 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc00ef-5a0a-4d24-8214-c0571e5e6c9f None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:21.462449 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca4cd8-b8c0-4fbc-84fc-e841e3a70feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:21.463661 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca4cd8-b8c0-4fbc-84fc-e841e3a70feb None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:21.463951 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca4cd8-b8c0-4fbc-84fc-e841e3a70feb None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:21.464225 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca4cd8-b8c0-4fbc-84fc-e841e3a70feb None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:21.465104 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ca4cd8-b8c0-4fbc-84fc-e841e3a70feb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:21.465416 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ca4cd8-b8c0-4fbc-84fc-e841e3a70feb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:21.466820 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ca4cd8-b8c0-4fbc-84fc-e841e3a70feb None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:21.477105 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01439da-568b-4442-b7f2-249ab315740d None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:21.484158 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 234/467] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:21.484158 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca4cd8-b8c0-4fbc-84fc-e841e3a70feb None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:21.521531 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b8c8ea-c44c-4e1d-97a2-f52a62018dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:21.522346 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8c8ea-c44c-4e1d-97a2-f52a62018dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:21.522632 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8c8ea-c44c-4e1d-97a2-f52a62018dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:21.522909 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8c8ea-c44c-4e1d-97a2-f52a62018dc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:21.536171 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b8c8ea-c44c-4e1d-97a2-f52a62018dc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:21.538596 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b8c8ea-c44c-4e1d-97a2-f52a62018dc2 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:21.560172 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-88b8c8ea-c44c-4e1d-97a2-f52a62018dc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:21.560172 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:21.560172 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:21.560172 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:21.560172 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:21.560172 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:21.560172 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:21.560172 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:21.560172 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:21.560172 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:21.560172 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:21.560172 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:21.560172 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:21.560172 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:21.560172 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:21.561303 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:21.561303 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:21.561303 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:21.561303 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:21.561303 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:21.561303 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:21.561303 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:21.561303 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:21.561303 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:21.561303 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:21.561303 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:21.561303 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:21.561303 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:21.561303 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:21.561303 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:21.561303 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:21.561303 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:21.562348 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:21.562348 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:21.562348 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:21.562348 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:21.563943 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 234/468] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:21.569541 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b8c8ea-c44c-4e1d-97a2-f52a62018dc2 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:21.599632 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca4cd8-b8c0-4fbc-84fc-e841e3a70feb None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:21.603067 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:21.635799 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5027ca2-79d1-4ef0-8369-a573b30fffc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:21.636690 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5027ca2-79d1-4ef0-8369-a573b30fffc7 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:21.636991 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5027ca2-79d1-4ef0-8369-a573b30fffc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:21.637261 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5027ca2-79d1-4ef0-8369-a573b30fffc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:21.649958 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5027ca2-79d1-4ef0-8369-a573b30fffc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:21.652062 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5027ca2-79d1-4ef0-8369-a573b30fffc7 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:21.672387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-e5027ca2-79d1-4ef0-8369-a573b30fffc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:21.672387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:21.672387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:21.672387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:21.672387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:21.672387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:21.672387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:21.672387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:21.672387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:21.672387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:21.672387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:21.672387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:21.672387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:21.672387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:21.672387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:21.673524 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:21.673524 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:21.673524 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:21.673524 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:21.673524 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:21.673524 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:21.673524 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:21.673524 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:21.673524 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:21.673524 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:21.673524 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:21.673524 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:21.673524 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:21.673524 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:21.673524 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:21.673524 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:21.673524 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:21.674426 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:21.674426 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:21.674426 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:21.674426 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:21.679368 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 235/469] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:21.680070 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5027ca2-79d1-4ef0-8369-a573b30fffc7 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:21.696281 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b8c8ea-c44c-4e1d-97a2-f52a62018dc2 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:21.698550 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:21.740077 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47997a6e-e9fc-4dc5-8231-3f03029e09a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:21.740928 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47997a6e-e9fc-4dc5-8231-3f03029e09a0 None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:21.741222 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47997a6e-e9fc-4dc5-8231-3f03029e09a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:21.741506 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47997a6e-e9fc-4dc5-8231-3f03029e09a0 None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:21.742230 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47997a6e-e9fc-4dc5-8231-3f03029e09a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:21.743716 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47997a6e-e9fc-4dc5-8231-3f03029e09a0 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:21.792181 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5027ca2-79d1-4ef0-8369-a573b30fffc7 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:21.827278 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1929af4f-8d8d-4c2f-91b7-326553344a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:21.827960 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929af4f-8d8d-4c2f-91b7-326553344a07 None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:21.828250 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929af4f-8d8d-4c2f-91b7-326553344a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:21.828572 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929af4f-8d8d-4c2f-91b7-326553344a07 None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:21.829298 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1929af4f-8d8d-4c2f-91b7-326553344a07 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:21.830741 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1929af4f-8d8d-4c2f-91b7-326553344a07 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:21.852578 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 235/470] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Jul 18 06:06:21 2022] POST /identity/v3/users => generated 246 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:06:21.853422 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47997a6e-e9fc-4dc5-8231-3f03029e09a0 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:21.985689 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47997a6e-e9fc-4dc5-8231-3f03029e09a0 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:22.027143 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901aa17f-f21a-4ff5-a171-c13e0bdbe0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:22.027382 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 236/471] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Jul 18 06:06:21 2022] POST /identity/v3/users => generated 248 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:06:22.027847 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901aa17f-f21a-4ff5-a171-c13e0bdbe0e3 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:22.028154 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901aa17f-f21a-4ff5-a171-c13e0bdbe0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:22.028506 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901aa17f-f21a-4ff5-a171-c13e0bdbe0e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:22.029280 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901aa17f-f21a-4ff5-a171-c13e0bdbe0e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:22.029408 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1929af4f-8d8d-4c2f-91b7-326553344a07 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:22.029767 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901aa17f-f21a-4ff5-a171-c13e0bdbe0e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:22.031031 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901aa17f-f21a-4ff5-a171-c13e0bdbe0e3 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:22.049713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:22.051160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 236/472] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:22.058360 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878160f7-8dd2-422f-84f6-5cd5ea92d8f2 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:22.058655 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878160f7-8dd2-422f-84f6-5cd5ea92d8f2 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:22.058937 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878160f7-8dd2-422f-84f6-5cd5ea92d8f2 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:22.060485 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 237/473] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:22 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:22.061292 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878160f7-8dd2-422f-84f6-5cd5ea92d8f2 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:22.085024 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:22.179911 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1929af4f-8d8d-4c2f-91b7-326553344a07 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:22.220029 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878160f7-8dd2-422f-84f6-5cd5ea92d8f2 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:22.227604 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de978afb-5125-4aa3-818b-0f1bb3ab5a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:22.228317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de978afb-5125-4aa3-818b-0f1bb3ab5a5e None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:22.230198 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de978afb-5125-4aa3-818b-0f1bb3ab5a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:22.230481 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de978afb-5125-4aa3-818b-0f1bb3ab5a5e None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:22.231239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de978afb-5125-4aa3-818b-0f1bb3ab5a5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:22.231553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de978afb-5125-4aa3-818b-0f1bb3ab5a5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:22.233433 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de978afb-5125-4aa3-818b-0f1bb3ab5a5e None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:22.248177 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 237/474] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:22.249702 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e4360-3ec5-4d42-8b70-dd91278fdafc None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:22.249998 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e4360-3ec5-4d42-8b70-dd91278fdafc None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:22.250278 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e4360-3ec5-4d42-8b70-dd91278fdafc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:22.271545 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-ff2e4360-3ec5-4d42-8b70-dd91278fdafc None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:22.272820 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ff2e4360-3ec5-4d42-8b70-dd91278fdafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:22.284178 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9912d9c-adda-4f80-b5ec-9d237ace6608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:22.285089 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9912d9c-adda-4f80-b5ec-9d237ace6608 None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:22.286353 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9912d9c-adda-4f80-b5ec-9d237ace6608 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:22.286353 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9912d9c-adda-4f80-b5ec-9d237ace6608 None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:22.287517 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9912d9c-adda-4f80-b5ec-9d237ace6608 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:22.289862 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9912d9c-adda-4f80-b5ec-9d237ace6608 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:22.381090 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 238/475] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:06:22.385937 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e4360-3ec5-4d42-8b70-dd91278fdafc None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:22.388864 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:22.410604 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 238/476] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Jul 18 06:06:22 2022] POST /identity/v3/users => generated 249 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:06:22.423550 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9912d9c-adda-4f80-b5ec-9d237ace6608 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:22.628749 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:22.641893 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e4360-3ec5-4d42-8b70-dd91278fdafc None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:22.671364 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9912d9c-adda-4f80-b5ec-9d237ace6608 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:22.696761 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7091d05-cd8f-4c07-81f1-b7493614dcfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:22.699310 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7091d05-cd8f-4c07-81f1-b7493614dcfa None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:22.699310 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7091d05-cd8f-4c07-81f1-b7493614dcfa None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:22.699310 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7091d05-cd8f-4c07-81f1-b7493614dcfa None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:22.699530 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7091d05-cd8f-4c07-81f1-b7493614dcfa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:22.700923 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7091d05-cd8f-4c07-81f1-b7493614dcfa None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:22.726570 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f9ae02-7267-4421-b199-75525344cffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:22.727596 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f9ae02-7267-4421-b199-75525344cffb None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:22.728077 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f9ae02-7267-4421-b199-75525344cffb None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:22.729288 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f9ae02-7267-4421-b199-75525344cffb None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:22.730459 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f9ae02-7267-4421-b199-75525344cffb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:22.732090 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f9ae02-7267-4421-b199-75525344cffb None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:22.769599 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.sql.core [None req-21f9ae02-7267-4421-b199-75525344cffb None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 18 06:06:22.769599 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 18 06:06:22.769599 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [parameters: {'id': 'b5dde83a48364795a4062de5be25eaa1', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 18 06:06:22.769599 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80470) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 18 06:06:22.776003 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 239/477] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Jul 18 06:06:22 2022] POST /identity/v3/users => generated 248 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:06:22.789721 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-21f9ae02-7267-4421-b199-75525344cffb None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 18 06:06:22.789721 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:22.789721 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 06:06:22.789721 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 06:06:22.789721 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 06:06:22.789721 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 06:06:22.789721 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 06:06:22.789721 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 06:06:22.789721 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 06:06:22.789721 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application conn.query(q) Jul 18 06:06:22.789721 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 06:06:22.789721 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 06:06:22.789721 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 06:06:22.789721 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application result.read() Jul 18 06:06:22.789721 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 06:06:22.791805 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 06:06:22.791805 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 06:06:22.791805 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 06:06:22.791805 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 06:06:22.791805 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 06:06:22.791805 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 06:06:22.791805 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 06:06:22.791805 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 18 06:06:22.791805 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:22.791805 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 18 06:06:22.791805 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:22.791805 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:22.791805 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 18 06:06:22.791805 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 18 06:06:22.791805 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 18 06:06:22.791805 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return ref.to_dict() Jul 18 06:06:22.791805 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 06:06:22.791805 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application next(self.gen) Jul 18 06:06:22.793287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 18 06:06:22.793287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application yield resource Jul 18 06:06:22.793287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 06:06:22.793287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application next(self.gen) Jul 18 06:06:22.793287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 18 06:06:22.793287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self.session.rollback() Jul 18 06:06:22.793287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 18 06:06:22.793287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self.force_reraise() Jul 18 06:06:22.793287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 18 06:06:22.793287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise self.value Jul 18 06:06:22.793287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 18 06:06:22.793287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 18 06:06:22.793287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 18 06:06:22.793287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application session.commit() Jul 18 06:06:22.793287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 18 06:06:22.793287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 18 06:06:22.793287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 18 06:06:22.794367 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._prepare_impl() Jul 18 06:06:22.794367 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 18 06:06:22.794367 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self.session.flush() Jul 18 06:06:22.794367 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 18 06:06:22.794367 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._flush(objects) Jul 18 06:06:22.794367 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 18 06:06:22.794367 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 18 06:06:22.794367 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 18 06:06:22.794367 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application compat.raise_( Jul 18 06:06:22.794367 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 06:06:22.794367 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception Jul 18 06:06:22.794367 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 18 06:06:22.794367 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application flush_context.execute() Jul 18 06:06:22.794367 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 18 06:06:22.794367 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rec.execute(self) Jul 18 06:06:22.794367 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 18 06:06:22.794367 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 18 06:06:22.795516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 18 06:06:22.795516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 18 06:06:22.795516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 18 06:06:22.795516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 18 06:06:22.795516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 18 06:06:22.795516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 18 06:06:22.795516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 18 06:06:22.795516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 18 06:06:22.795516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 18 06:06:22.795516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 18 06:06:22.795516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 18 06:06:22.795516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 18 06:06:22.795516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 18 06:06:22.795516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 18 06:06:22.795516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 06:06:22.795516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception Jul 18 06:06:22.796636 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 06:06:22.796636 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 06:06:22.796636 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 06:06:22.796636 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 06:06:22.796636 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 06:06:22.796636 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 06:06:22.796636 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 06:06:22.796636 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application conn.query(q) Jul 18 06:06:22.796636 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 06:06:22.796636 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 06:06:22.796636 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 06:06:22.796636 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application result.read() Jul 18 06:06:22.796636 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 06:06:22.796636 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 06:06:22.796636 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 06:06:22.796636 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 06:06:22.797663 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 06:06:22.797663 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 06:06:22.797663 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 06:06:22.797663 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 06:06:22.797663 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 18 06:06:22.797663 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 18 06:06:22.797663 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [parameters: {'id': 'b5dde83a48364795a4062de5be25eaa1', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 18 06:06:22.797663 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 18 06:06:22.797663 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:22.797663 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 06:06:22.797663 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:22.797663 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:22.797663 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:22.797663 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:22.797663 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:22.797663 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:22.798702 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:22.798702 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:22.798702 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:22.798702 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:22.798702 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:22.798702 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:22.798702 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 18 06:06:22.798702 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 18 06:06:22.798702 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:22.798702 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:22.798702 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 18 06:06:22.798702 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 18 06:06:22.798702 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 18 06:06:22.798702 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 18 06:06:22.798702 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 18 06:06:22.798702 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:22.801837 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 239/478] 10.222.0.40 () {60 vars in 1131 bytes} [Mon Jul 18 06:06:22 2022] POST /identity/v3/roles => generated 144 bytes in 379 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:06:22.808590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7091d05-cd8f-4c07-81f1-b7493614dcfa None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:22.883002 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:22.943814 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7091d05-cd8f-4c07-81f1-b7493614dcfa None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:22.994082 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8afcd3-3dbb-4068-9047-9e5de457f877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:22.994941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8afcd3-3dbb-4068-9047-9e5de457f877 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:22.995234 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8afcd3-3dbb-4068-9047-9e5de457f877 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:22.995519 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8afcd3-3dbb-4068-9047-9e5de457f877 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:23.013123 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8afcd3-3dbb-4068-9047-9e5de457f877 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:23.021491 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-0f8afcd3-3dbb-4068-9047-9e5de457f877 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 06:06:23.021491 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:23.021491 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:23.021491 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:23.021491 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:23.021491 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:23.021491 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:23.021491 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:23.021491 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:23.021491 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:23.021491 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:23.021491 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:23.021491 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:23.021491 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:23.021491 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:23.021491 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:23.022400 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:23.022400 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:23.022400 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:23.022400 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:23.022400 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:23.022400 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:23.022400 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:23.022400 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:23.022400 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:23.022400 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:23.022400 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:23.022400 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:23.022400 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:23.022400 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:23.022400 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:23.022400 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:23.023201 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:23.023201 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:23.023201 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:23.023201 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:23.023201 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:23.023201 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:23.023201 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:23.023201 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:23.023201 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:23.023201 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:23.023201 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 06:06:23.023201 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:23.023201 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 240/479] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 18 06:06:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:23.029430 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f9ae02-7267-4421-b199-75525344cffb None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:23.153780 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f9ae02-7267-4421-b199-75525344cffb None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:23.191677 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914bb2c-8dad-4934-a864-a1ab5076bdfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:23.192690 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914bb2c-8dad-4934-a864-a1ab5076bdfd None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:23.193183 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914bb2c-8dad-4934-a864-a1ab5076bdfd None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:23.193644 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914bb2c-8dad-4934-a864-a1ab5076bdfd None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:23.194676 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3914bb2c-8dad-4934-a864-a1ab5076bdfd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:23.195179 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3914bb2c-8dad-4934-a864-a1ab5076bdfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:23.196807 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3914bb2c-8dad-4934-a864-a1ab5076bdfd None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:23.203765 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:23.209173 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 240/480] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:06:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:24.252601 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeac378-9c66-413e-a9d8-bb672e181007 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:24.253119 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeac378-9c66-413e-a9d8-bb672e181007 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:24.253194 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeac378-9c66-413e-a9d8-bb672e181007 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:24.254816 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 241/481] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:24.260469 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57161fa-d199-4f86-8c4b-f60d6324b62d None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:24.260983 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57161fa-d199-4f86-8c4b-f60d6324b62d None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:24.261062 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57161fa-d199-4f86-8c4b-f60d6324b62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:24.283109 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-d57161fa-d199-4f86-8c4b-f60d6324b62d None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:24.284430 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d57161fa-d199-4f86-8c4b-f60d6324b62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:24.381271 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 241/482] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:06:24.417667 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeac378-9c66-413e-a9d8-bb672e181007 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:24.562332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeac378-9c66-413e-a9d8-bb672e181007 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:24.597166 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:24.617329 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b166c8fd-20b4-4226-b665-05521961198b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:24.618137 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166c8fd-20b4-4226-b665-05521961198b None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:24.618420 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166c8fd-20b4-4226-b665-05521961198b None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:24.624016 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166c8fd-20b4-4226-b665-05521961198b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:24.641906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b166c8fd-20b4-4226-b665-05521961198b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:24.649951 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-b166c8fd-20b4-4226-b665-05521961198b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:06:24.649951 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:24.649951 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:24.649951 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:24.649951 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:24.649951 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:24.649951 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:24.649951 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:24.649951 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:24.649951 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:24.649951 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:24.649951 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:24.649951 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:24.649951 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:24.649951 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:24.649951 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:24.651530 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:24.651530 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:24.651530 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:24.651530 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:24.651530 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:24.651530 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:24.651530 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:24.651530 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:24.651530 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:24.651530 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:24.651530 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:24.651530 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:24.651530 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:24.651530 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:24.651530 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:24.651530 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:24.652506 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:24.652506 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:24.652506 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:24.652506 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:24.652506 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:24.652506 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:24.652506 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:24.652506 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:24.652506 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:24.652506 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:24.652506 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:06:24.652506 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:24.662920 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 242/483] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 18 06:06:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:24.668233 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57161fa-d199-4f86-8c4b-f60d6324b62d None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:24.706399 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0109d-0ff1-43ab-ac4c-0e52e6ae87cb None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:24.706730 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0109d-0ff1-43ab-ac4c-0e52e6ae87cb None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:24.706951 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0109d-0ff1-43ab-ac4c-0e52e6ae87cb None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:24.708649 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 243/484] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:24.714987 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371de8a0-1c3e-4113-94a7-39f9954a7c5c None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:24.715273 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371de8a0-1c3e-4113-94a7-39f9954a7c5c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:24.715548 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371de8a0-1c3e-4113-94a7-39f9954a7c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:24.755430 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-371de8a0-1c3e-4113-94a7-39f9954a7c5c None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:24.756660 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-371de8a0-1c3e-4113-94a7-39f9954a7c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:24.798428 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57161fa-d199-4f86-8c4b-f60d6324b62d None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:24.852992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:24.872679 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 244/485] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 06:06:24.885810 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583a0e3-8099-4655-9ed8-a8da299e6dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:24.886646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583a0e3-8099-4655-9ed8-a8da299e6dac None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:24.886937 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583a0e3-8099-4655-9ed8-a8da299e6dac None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:24.887225 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583a0e3-8099-4655-9ed8-a8da299e6dac None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:24.888027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b583a0e3-8099-4655-9ed8-a8da299e6dac None admin] RBAC: Adding query filter params (name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:24.888338 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b583a0e3-8099-4655-9ed8-a8da299e6dac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:24.889860 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b583a0e3-8099-4655-9ed8-a8da299e6dac None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:24.893394 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371de8a0-1c3e-4113-94a7-39f9954a7c5c None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:24.908325 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 242/486] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 18 06:06:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:24.914314 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583a0e3-8099-4655-9ed8-a8da299e6dac None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:25.005487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371de8a0-1c3e-4113-94a7-39f9954a7c5c None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:25.032815 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:25.045868 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff31f2ce-b012-4820-b67a-b5a1055eb19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:25.047604 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31f2ce-b012-4820-b67a-b5a1055eb19a None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:25.047604 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31f2ce-b012-4820-b67a-b5a1055eb19a None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:25.047604 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31f2ce-b012-4820-b67a-b5a1055eb19a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:25.060281 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff31f2ce-b012-4820-b67a-b5a1055eb19a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:25.072292 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-ff31f2ce-b012-4820-b67a-b5a1055eb19a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:06:25.072292 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:25.072292 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:25.072292 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:25.072292 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:25.072292 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:25.072292 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:25.072292 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:25.072292 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:25.072292 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:25.072292 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:25.072292 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:25.072292 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:25.072292 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:25.072292 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:25.072292 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:25.073477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:25.073477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:25.073477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:25.073477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:25.073477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:25.073477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:25.073477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:25.073477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:25.073477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:25.073477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:25.073477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:25.073477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:25.073477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:25.073477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:25.073477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:25.073477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:25.074346 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:25.074346 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:25.074346 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:25.074346 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:25.074346 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:25.074346 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:25.074346 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:25.074346 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:25.074346 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:25.074346 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:25.074346 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:06:25.074346 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:25.078810 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 245/487] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 18 06:06:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:06:25.080368 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff31f2ce-b012-4820-b67a-b5a1055eb19a None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:25.162693 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583a0e3-8099-4655-9ed8-a8da299e6dac None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:25.200679 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff31f2ce-b012-4820-b67a-b5a1055eb19a None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:25.239400 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3fc7d3-3175-431b-9e1e-fd30ec612a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:25.240074 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3fc7d3-3175-431b-9e1e-fd30ec612a97 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:25.240393 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3fc7d3-3175-431b-9e1e-fd30ec612a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:25.240696 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3fc7d3-3175-431b-9e1e-fd30ec612a97 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:25.241449 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3fc7d3-3175-431b-9e1e-fd30ec612a97 None admin] RBAC: Adding query filter params (name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:25.242035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3fc7d3-3175-431b-9e1e-fd30ec612a97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:25.243453 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3fc7d3-3175-431b-9e1e-fd30ec612a97 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:25.258064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18520dc4-f40d-4407-ab84-fd83d75d32d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:25.258875 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18520dc4-f40d-4407-ab84-fd83d75d32d5 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:25.259162 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18520dc4-f40d-4407-ab84-fd83d75d32d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:25.259442 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18520dc4-f40d-4407-ab84-fd83d75d32d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:25.262578 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 246/488] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 18 06:06:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:25.268306 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3fc7d3-3175-431b-9e1e-fd30ec612a97 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:25.286349 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18520dc4-f40d-4407-ab84-fd83d75d32d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:25.288100 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18520dc4-f40d-4407-ab84-fd83d75d32d5 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:25.321341 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-18520dc4-f40d-4407-ab84-fd83d75d32d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:25.321341 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:25.321341 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:25.321341 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:25.321341 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:25.321341 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:25.321341 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:25.321341 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:25.321341 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:25.321341 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:25.321341 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:25.321341 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:25.321341 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:25.321341 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:25.321341 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:25.322363 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:25.322363 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:25.322363 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:25.322363 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:25.322363 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:25.322363 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:25.322363 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:25.322363 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:25.322363 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:25.322363 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:25.322363 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:25.322363 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:25.322363 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:25.322363 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:25.322363 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:25.322363 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:25.322363 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:25.323394 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:25.323394 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:25.323394 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:25.323394 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:25.323394 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 243/489] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 409 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:06:25.324281 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b5071-4ad5-4d23-b9cd-6b9075a5d3ca None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:25.324595 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b5071-4ad5-4d23-b9cd-6b9075a5d3ca None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:25.324876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b5071-4ad5-4d23-b9cd-6b9075a5d3ca None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:25.331684 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 244/490] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:25.331684 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b5071-4ad5-4d23-b9cd-6b9075a5d3ca None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:25.331922 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:25.389100 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:25.414393 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3fc7d3-3175-431b-9e1e-fd30ec612a97 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:25.449782 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b5071-4ad5-4d23-b9cd-6b9075a5d3ca None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:25.454290 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec36c79e-cf32-4cba-a204-6fd2cb7d69e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:25.455349 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36c79e-cf32-4cba-a204-6fd2cb7d69e7 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:25.455899 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36c79e-cf32-4cba-a204-6fd2cb7d69e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:25.456383 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36c79e-cf32-4cba-a204-6fd2cb7d69e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:25.469794 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec36c79e-cf32-4cba-a204-6fd2cb7d69e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:25.471868 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec36c79e-cf32-4cba-a204-6fd2cb7d69e7 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:25.487249 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2050a9ad-47c4-403b-828d-64a97b6b4eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:25.487924 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050a9ad-47c4-403b-828d-64a97b6b4eea None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:25.488214 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050a9ad-47c4-403b-828d-64a97b6b4eea None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:25.488539 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050a9ad-47c4-403b-828d-64a97b6b4eea None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:25.489276 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2050a9ad-47c4-403b-828d-64a97b6b4eea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:25.489591 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2050a9ad-47c4-403b-828d-64a97b6b4eea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:25.490986 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2050a9ad-47c4-403b-828d-64a97b6b4eea None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:25.494663 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-ec36c79e-cf32-4cba-a204-6fd2cb7d69e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:25.494663 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:25.494663 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:25.494663 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:25.494663 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:25.494663 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:25.494663 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:25.494663 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:25.494663 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:25.494663 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:25.494663 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:25.494663 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:25.494663 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:25.494663 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:25.494663 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:25.495553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:25.495553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:25.495553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:25.495553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:25.495553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:25.495553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:25.495553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:25.495553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:25.495553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:25.495553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:25.495553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:25.495553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:25.495553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:25.495553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:25.495553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:25.495553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:25.495553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:25.496489 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:25.496489 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:25.496489 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:25.496489 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:25.498771 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 247/491] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:06:25.500534 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98001f56-9586-4bef-9dd5-a48fce8ca02c None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:25.501173 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98001f56-9586-4bef-9dd5-a48fce8ca02c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:25.501653 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98001f56-9586-4bef-9dd5-a48fce8ca02c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:25.508463 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 245/492] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:25.508463 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2050a9ad-47c4-403b-828d-64a97b6b4eea None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:25.524666 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-98001f56-9586-4bef-9dd5-a48fce8ca02c None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:25.526276 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-98001f56-9586-4bef-9dd5-a48fce8ca02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:25.604114 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 248/493] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:06:25.605186 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98001f56-9586-4bef-9dd5-a48fce8ca02c None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:25.622454 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:25.625061 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2050a9ad-47c4-403b-828d-64a97b6b4eea None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:25.627806 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:25.664836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5b4872-e802-4387-8f48-7a077db590fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:25.665592 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b4872-e802-4387-8f48-7a077db590fa None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:25.666010 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b4872-e802-4387-8f48-7a077db590fa None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:25.666303 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b4872-e802-4387-8f48-7a077db590fa None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:25.667099 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5b4872-e802-4387-8f48-7a077db590fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:25.667469 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5b4872-e802-4387-8f48-7a077db590fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:25.668944 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5b4872-e802-4387-8f48-7a077db590fa None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:25.692441 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 246/494] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:25.692441 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2a9ca-949d-41b5-8dd9-04e853604827 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:25.692441 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2a9ca-949d-41b5-8dd9-04e853604827 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:25.692441 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2a9ca-949d-41b5-8dd9-04e853604827 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:25.692441 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 247/495] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:25.693722 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2a9ca-949d-41b5-8dd9-04e853604827 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:25.726839 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98001f56-9586-4bef-9dd5-a48fce8ca02c None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:25.765365 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0068da3a-b877-43d7-b24d-a4144f3835a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:25.766169 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0068da3a-b877-43d7-b24d-a4144f3835a7 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:25.766460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0068da3a-b877-43d7-b24d-a4144f3835a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:25.766743 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0068da3a-b877-43d7-b24d-a4144f3835a7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:25.775944 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0068da3a-b877-43d7-b24d-a4144f3835a7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:25.777475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0068da3a-b877-43d7-b24d-a4144f3835a7 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:25.796673 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-0068da3a-b877-43d7-b24d-a4144f3835a7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 06:06:25.796673 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:25.796673 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:25.796673 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:25.796673 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:25.796673 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:25.796673 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:25.796673 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:25.796673 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:25.796673 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:25.796673 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:25.796673 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:25.796673 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 06:06:25.796673 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 06:06:25.796673 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 06:06:25.796673 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 06:06:25.798118 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:25.798118 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:25.798118 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 06:06:25.798118 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:25.798118 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 06:06:25.798118 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:25.798118 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:25.798118 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:25.798118 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:25.798118 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:25.798118 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:25.798118 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:25.798118 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:25.798118 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:25.798118 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:25.798118 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:25.798118 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:25.798118 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:25.799017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:25.799017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:25.799017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 06:06:25.799017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 06:06:25.799017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 06:06:25.799017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 06:06:25.799017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 06:06:25.799017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 06:06:25.799017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 06:06:25.799017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:25.799017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 249/496] 10.222.0.40 () {56 vars in 1093 bytes} [Mon Jul 18 06:06:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:25.799017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0068da3a-b877-43d7-b24d-a4144f3835a7 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:25.813043 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2a9ca-949d-41b5-8dd9-04e853604827 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:25.849445 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64fee86-7451-4f9f-8537-796a53ee0ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:25.850269 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64fee86-7451-4f9f-8537-796a53ee0ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:25.850549 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64fee86-7451-4f9f-8537-796a53ee0ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:25.850830 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64fee86-7451-4f9f-8537-796a53ee0ac0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:25.864168 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64fee86-7451-4f9f-8537-796a53ee0ac0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:25.872761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-e64fee86-7451-4f9f-8537-796a53ee0ac0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:06:25.872761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:25.872761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:25.872761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:25.872761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:25.872761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:25.872761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:25.872761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:25.872761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:25.872761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:25.872761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:25.872761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:25.872761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:25.872761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:25.872761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:25.872761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:25.873926 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:25.873926 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:25.873926 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:25.873926 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:25.873926 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:25.873926 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:25.873926 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:25.873926 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:25.873926 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:25.873926 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:25.873926 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:25.873926 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:25.873926 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:25.873926 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:25.873926 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:25.873926 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:25.874760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:25.874760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:25.874760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:25.874760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:25.874760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:25.874760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:25.874760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:25.874760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:25.874760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:25.874760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:25.874760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:06:25.874760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:25.874760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 248/497] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 18 06:06:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:25.875413 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5a950-446a-458e-a722-9bf7096aa3ef None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:25.875609 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5a950-446a-458e-a722-9bf7096aa3ef None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:25.875914 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5a950-446a-458e-a722-9bf7096aa3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:25.891270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:25.893842 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-e4a5a950-446a-458e-a722-9bf7096aa3ef None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:25.895061 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e4a5a950-446a-458e-a722-9bf7096aa3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:25.914411 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:25.928883 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0068da3a-b877-43d7-b24d-a4144f3835a7 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:25.965861 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7cbac4-c22b-4184-b82d-0be3d1b8c4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:25.966659 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7cbac4-c22b-4184-b82d-0be3d1b8c4e8 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:25.966947 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7cbac4-c22b-4184-b82d-0be3d1b8c4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:25.967293 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7cbac4-c22b-4184-b82d-0be3d1b8c4e8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:25.968145 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 249/498] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:06:25.969023 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5a950-446a-458e-a722-9bf7096aa3ef None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:25.986805 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7cbac4-c22b-4184-b82d-0be3d1b8c4e8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:25.988353 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7cbac4-c22b-4184-b82d-0be3d1b8c4e8 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:26.003906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-3d7cbac4-c22b-4184-b82d-0be3d1b8c4e8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 06:06:26.003906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:26.003906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:26.003906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:26.003906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:26.003906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:26.003906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:26.003906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:26.003906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:26.003906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:26.003906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:26.003906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:26.003906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 06:06:26.003906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 06:06:26.003906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 06:06:26.003906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 06:06:26.005012 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:26.005012 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:26.005012 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 06:06:26.005012 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:26.005012 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 06:06:26.005012 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:26.005012 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:26.005012 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:26.005012 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:26.005012 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:26.005012 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:26.005012 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:26.005012 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:26.005012 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:26.005012 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:26.005012 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:26.005012 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:26.005012 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:26.005939 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:26.005939 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:26.005939 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 06:06:26.005939 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 06:06:26.005939 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 06:06:26.005939 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 06:06:26.005939 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 06:06:26.005939 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 06:06:26.005939 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 06:06:26.005939 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:26.005939 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 250/499] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:06:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:26.005939 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7cbac4-c22b-4184-b82d-0be3d1b8c4e8 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:26.082174 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5a950-446a-458e-a722-9bf7096aa3ef None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:26.111666 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7cbac4-c22b-4184-b82d-0be3d1b8c4e8 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:26.117322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9e08b-66e6-4897-99c4-7d7dda73bcdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:26.118002 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9e08b-66e6-4897-99c4-7d7dda73bcdc None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:26.118288 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9e08b-66e6-4897-99c4-7d7dda73bcdc None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:26.118569 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9e08b-66e6-4897-99c4-7d7dda73bcdc None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:26.119453 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba9e08b-66e6-4897-99c4-7d7dda73bcdc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:26.119783 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba9e08b-66e6-4897-99c4-7d7dda73bcdc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:26.121265 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba9e08b-66e6-4897-99c4-7d7dda73bcdc None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:26.141711 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:26.146554 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa11947-5b7c-4b17-8cd1-c9e64882c709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:26.147391 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa11947-5b7c-4b17-8cd1-c9e64882c709 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:26.147678 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa11947-5b7c-4b17-8cd1-c9e64882c709 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:26.147956 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa11947-5b7c-4b17-8cd1-c9e64882c709 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:26.148748 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa11947-5b7c-4b17-8cd1-c9e64882c709 None admin] RBAC: Adding query filter params (name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:26.149062 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa11947-5b7c-4b17-8cd1-c9e64882c709 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:26.151074 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa11947-5b7c-4b17-8cd1-c9e64882c709 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:26.159435 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 251/500] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 18 06:06:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:26.160552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 250/501] 10.222.0.40 () {56 vars in 1133 bytes} [Mon Jul 18 06:06:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:26.160741 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4d6a4-ae49-40e7-ae3b-dc85ff96e61d None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:26.161031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4d6a4-ae49-40e7-ae3b-dc85ff96e61d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:26.161310 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4d6a4-ae49-40e7-ae3b-dc85ff96e61d None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:26.161400 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9e08b-66e6-4897-99c4-7d7dda73bcdc None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:26.164638 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 252/502] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:26.164638 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4d6a4-ae49-40e7-ae3b-dc85ff96e61d None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:26.178508 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:26.267072 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9e08b-66e6-4897-99c4-7d7dda73bcdc None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:26.268572 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4d6a4-ae49-40e7-ae3b-dc85ff96e61d None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:26.303069 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388aa529-e0cd-488a-a724-dfe9100d8569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:26.303619 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2080eb-6d12-4d5d-a536-d9de34b1ff1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:26.303874 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388aa529-e0cd-488a-a724-dfe9100d8569 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:26.303967 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2080eb-6d12-4d5d-a536-d9de34b1ff1c None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:26.304161 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2080eb-6d12-4d5d-a536-d9de34b1ff1c None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:26.304245 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388aa529-e0cd-488a-a724-dfe9100d8569 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:26.304469 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388aa529-e0cd-488a-a724-dfe9100d8569 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:26.304619 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2080eb-6d12-4d5d-a536-d9de34b1ff1c None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:26.305315 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2080eb-6d12-4d5d-a536-d9de34b1ff1c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:26.305624 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2080eb-6d12-4d5d-a536-d9de34b1ff1c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:26.307089 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2080eb-6d12-4d5d-a536-d9de34b1ff1c None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:26.314854 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388aa529-e0cd-488a-a724-dfe9100d8569 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:26.323508 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-388aa529-e0cd-488a-a724-dfe9100d8569 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:06:26.323508 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:26.323508 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:26.323508 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:26.323508 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:26.323508 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:26.323508 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:26.323508 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:26.323508 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:26.323508 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:26.323508 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:26.323508 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:26.323508 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:26.323508 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:26.323508 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:26.323508 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:26.324464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:26.324464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:26.324464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:26.324464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:26.324464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:26.324464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:26.324464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:26.324464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:26.324464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:26.324464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:26.324464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:26.324464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:26.324464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:26.324464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:26.324464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:26.324464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:26.325304 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:26.325304 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:26.325304 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:26.325304 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:26.325304 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:26.325304 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:26.325304 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:26.325304 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:26.325304 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:26.325304 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:26.325304 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:06:26.325304 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:26.325304 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 251/503] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 18 06:06:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:26.325968 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388aa529-e0cd-488a-a724-dfe9100d8569 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:26.346406 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 253/504] 10.222.0.40 () {56 vars in 1137 bytes} [Mon Jul 18 06:06:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:26.347687 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a702e8-e767-4869-8709-26ec98ee84b1 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:26.347968 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a702e8-e767-4869-8709-26ec98ee84b1 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:26.348245 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a702e8-e767-4869-8709-26ec98ee84b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:26.364594 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-65a702e8-e767-4869-8709-26ec98ee84b1 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:26.365788 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-65a702e8-e767-4869-8709-26ec98ee84b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:26.404080 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:26.416129 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:26.434484 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388aa529-e0cd-488a-a724-dfe9100d8569 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:26.435620 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 254/505] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:06:26.437398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a702e8-e767-4869-8709-26ec98ee84b1 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:26.471454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5be297-20a0-461a-b648-fdcd9f400ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:26.472253 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5be297-20a0-461a-b648-fdcd9f400ffd None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:26.472567 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5be297-20a0-461a-b648-fdcd9f400ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:26.472843 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5be297-20a0-461a-b648-fdcd9f400ffd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:26.483761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5be297-20a0-461a-b648-fdcd9f400ffd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:26.485465 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5be297-20a0-461a-b648-fdcd9f400ffd None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:26.503985 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-6d5be297-20a0-461a-b648-fdcd9f400ffd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:26.503985 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:26.503985 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:26.503985 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:26.503985 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:26.503985 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:26.503985 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:26.503985 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:26.503985 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:26.503985 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:26.503985 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:26.503985 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:26.503985 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:26.503985 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:26.503985 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:26.505083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:26.505083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:26.505083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:26.505083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:26.505083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:26.505083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:26.505083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:26.505083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:26.505083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:26.505083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:26.505083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:26.505083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:26.505083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:26.505083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:26.505083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:26.505083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:26.505083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:26.505960 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:26.505960 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:26.505960 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:26.505960 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:26.505960 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 252/506] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:26.506231 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5be297-20a0-461a-b648-fdcd9f400ffd None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:26.543145 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a702e8-e767-4869-8709-26ec98ee84b1 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:26.581049 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69071623-f271-47c9-a714-8691f18eee8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:26.581706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69071623-f271-47c9-a714-8691f18eee8a None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:26.582059 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69071623-f271-47c9-a714-8691f18eee8a None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:26.582264 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69071623-f271-47c9-a714-8691f18eee8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:26.625608 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5be297-20a0-461a-b648-fdcd9f400ffd None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:26.644587 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:26.649736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:26.660490 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2cb5b4-e077-4cbd-b75c-5af331f1f21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:26.661172 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2cb5b4-e077-4cbd-b75c-5af331f1f21d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:26.661462 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2cb5b4-e077-4cbd-b75c-5af331f1f21d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:26.661744 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2cb5b4-e077-4cbd-b75c-5af331f1f21d None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:26.662832 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2cb5b4-e077-4cbd-b75c-5af331f1f21d None admin] RBAC: Adding query filter params (name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:26.663146 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2cb5b4-e077-4cbd-b75c-5af331f1f21d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:26.663527 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69071623-f271-47c9-a714-8691f18eee8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:26.664582 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2cb5b4-e077-4cbd-b75c-5af331f1f21d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:26.665047 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69071623-f271-47c9-a714-8691f18eee8a None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:26.673799 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 253/507] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 18 06:06:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:26.674654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2cb5b4-e077-4cbd-b75c-5af331f1f21d None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:26.768680 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 255/508] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:06:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 06:06:26.769524 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69071623-f271-47c9-a714-8691f18eee8a None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:26.786117 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2cb5b4-e077-4cbd-b75c-5af331f1f21d None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:26.821466 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b27e4b-a378-4c25-883e-73c0fc97053a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:26.822545 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b27e4b-a378-4c25-883e-73c0fc97053a None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:26.823077 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b27e4b-a378-4c25-883e-73c0fc97053a None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:26.823077 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b27e4b-a378-4c25-883e-73c0fc97053a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:26.857648 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:26.876261 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69071623-f271-47c9-a714-8691f18eee8a None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:26.879569 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:26.893479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b27e4b-a378-4c25-883e-73c0fc97053a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:26.895017 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b27e4b-a378-4c25-883e-73c0fc97053a None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:26.912672 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d3a9d-7821-458c-91fa-26ed37289503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:26.913735 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d3a9d-7821-458c-91fa-26ed37289503 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:26.914304 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d3a9d-7821-458c-91fa-26ed37289503 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:26.914580 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d3a9d-7821-458c-91fa-26ed37289503 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:26.925100 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30d3a9d-7821-458c-91fa-26ed37289503 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:26.926690 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30d3a9d-7821-458c-91fa-26ed37289503 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:26.944179 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-c30d3a9d-7821-458c-91fa-26ed37289503 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:26.944179 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:26.944179 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:26.944179 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:26.944179 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:26.944179 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:26.944179 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:26.944179 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:26.944179 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:26.944179 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:26.944179 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:26.944179 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:26.944179 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:26.944179 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:26.944179 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:26.945111 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:26.945111 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:26.945111 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:26.945111 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:26.945111 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:26.945111 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:26.945111 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:26.945111 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:26.945111 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:26.945111 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:26.945111 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:26.945111 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:26.945111 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:26.945111 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:26.945111 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:26.945111 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:26.945111 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:26.945959 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:26.945959 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:26.945959 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:26.945959 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:26.945959 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 256/509] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:26.946535 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d3a9d-7821-458c-91fa-26ed37289503 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:26.999998 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 254/510] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:06:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 06:06:27.000907 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b27e4b-a378-4c25-883e-73c0fc97053a None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:27.047965 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d3a9d-7821-458c-91fa-26ed37289503 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:27.082619 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa63604-8ee7-4e49-b6c1-a57ff6b77202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:27.083196 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa63604-8ee7-4e49-b6c1-a57ff6b77202 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:27.083484 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa63604-8ee7-4e49-b6c1-a57ff6b77202 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:27.083762 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa63604-8ee7-4e49-b6c1-a57ff6b77202 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:27.084632 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa63604-8ee7-4e49-b6c1-a57ff6b77202 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:27.084946 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa63604-8ee7-4e49-b6c1-a57ff6b77202 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:27.086331 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa63604-8ee7-4e49-b6c1-a57ff6b77202 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:27.088572 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:27.089178 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:27.099325 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 257/511] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:27.100140 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa63604-8ee7-4e49-b6c1-a57ff6b77202 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:27.105796 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b27e4b-a378-4c25-883e-73c0fc97053a None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:27.140648 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d573ca30-c937-4b96-89e2-56f0908a190e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:27.141448 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573ca30-c937-4b96-89e2-56f0908a190e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:27.141727 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573ca30-c937-4b96-89e2-56f0908a190e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:27.142005 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573ca30-c937-4b96-89e2-56f0908a190e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:27.152046 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d573ca30-c937-4b96-89e2-56f0908a190e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:27.153530 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d573ca30-c937-4b96-89e2-56f0908a190e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:27.172187 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-d573ca30-c937-4b96-89e2-56f0908a190e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:27.172187 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:27.172187 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:27.172187 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:27.172187 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:27.172187 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:27.172187 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:27.172187 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:27.172187 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:27.172187 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:27.172187 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:27.172187 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:27.172187 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:27.172187 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:27.172187 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:27.173272 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:27.173272 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:27.173272 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:27.173272 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:27.173272 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:27.173272 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:27.173272 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:27.173272 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:27.173272 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:27.173272 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:27.173272 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:27.173272 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:27.173272 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:27.173272 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:27.173272 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:27.173272 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:27.173272 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:27.174256 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:27.174256 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:27.174256 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:27.174256 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:27.174256 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 255/512] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:27.174256 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d573ca30-c937-4b96-89e2-56f0908a190e None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:27.213358 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa63604-8ee7-4e49-b6c1-a57ff6b77202 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:27.246814 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1595c924-a10e-4d46-ad1f-ca772d48187d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:27.247601 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595c924-a10e-4d46-ad1f-ca772d48187d None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:27.247882 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595c924-a10e-4d46-ad1f-ca772d48187d None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:27.248162 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595c924-a10e-4d46-ad1f-ca772d48187d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:27.258433 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1595c924-a10e-4d46-ad1f-ca772d48187d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:27.260548 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1595c924-a10e-4d46-ad1f-ca772d48187d None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:27.275944 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d573ca30-c937-4b96-89e2-56f0908a190e None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:27.278790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-1595c924-a10e-4d46-ad1f-ca772d48187d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:27.278790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:27.278790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:27.278790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:27.278790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:27.278790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:27.278790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:27.278790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:27.278790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:27.278790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:27.278790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:27.278790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:27.278790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:27.278790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:27.278790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:27.280162 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:27.280162 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:27.280162 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:27.280162 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:27.280162 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:27.280162 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:27.280162 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:27.280162 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:27.280162 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:27.280162 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:27.280162 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:27.280162 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:27.280162 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:27.280162 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:27.280162 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:27.280162 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:27.280162 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:27.281583 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:27.281583 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:27.281583 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:27.281583 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:27.281583 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 258/513] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:27.281583 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1595c924-a10e-4d46-ad1f-ca772d48187d None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:27.314215 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5043897-aa7e-449b-9ba3-bdddaacb0801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:27.314885 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5043897-aa7e-449b-9ba3-bdddaacb0801 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:27.315171 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5043897-aa7e-449b-9ba3-bdddaacb0801 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:27.315454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5043897-aa7e-449b-9ba3-bdddaacb0801 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:27.316180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5043897-aa7e-449b-9ba3-bdddaacb0801 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:27.316522 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5043897-aa7e-449b-9ba3-bdddaacb0801 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:27.317912 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5043897-aa7e-449b-9ba3-bdddaacb0801 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:27.325577 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:27.330799 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 256/514] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:27.331603 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5043897-aa7e-449b-9ba3-bdddaacb0801 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:27.335056 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:27.388517 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1595c924-a10e-4d46-ad1f-ca772d48187d None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:27.427021 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf8368-3a43-4c19-a3ca-d6052dbb6da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:27.427822 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf8368-3a43-4c19-a3ca-d6052dbb6da0 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:27.428102 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf8368-3a43-4c19-a3ca-d6052dbb6da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:27.428383 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf8368-3a43-4c19-a3ca-d6052dbb6da0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:27.437336 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5043897-aa7e-449b-9ba3-bdddaacb0801 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:27.439780 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdf8368-3a43-4c19-a3ca-d6052dbb6da0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:27.441267 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdf8368-3a43-4c19-a3ca-d6052dbb6da0 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:27.459544 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-6fdf8368-3a43-4c19-a3ca-d6052dbb6da0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:27.459544 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:27.459544 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:27.459544 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:27.459544 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:27.459544 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:27.459544 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:27.459544 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:27.459544 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:27.459544 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:27.459544 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:27.459544 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:27.459544 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:27.459544 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:27.459544 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:27.460460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:27.460460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:27.460460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:27.460460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:27.460460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:27.460460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:27.460460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:27.460460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:27.460460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:27.460460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:27.460460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:27.460460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:27.460460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:27.460460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:27.460460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:27.460460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:27.460460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:27.461356 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:27.461356 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:27.461356 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:27.461356 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:27.461356 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 259/515] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:27.461626 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf8368-3a43-4c19-a3ca-d6052dbb6da0 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:27.477157 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63db764-f7f5-41ac-a64f-e18b44cab389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:27.477972 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63db764-f7f5-41ac-a64f-e18b44cab389 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:27.478854 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63db764-f7f5-41ac-a64f-e18b44cab389 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:27.479202 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63db764-f7f5-41ac-a64f-e18b44cab389 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:27.487140 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63db764-f7f5-41ac-a64f-e18b44cab389 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:27.488659 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63db764-f7f5-41ac-a64f-e18b44cab389 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:27.505429 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-d63db764-f7f5-41ac-a64f-e18b44cab389 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 06:06:27.505429 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:27.505429 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:27.505429 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:27.505429 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:27.505429 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:27.505429 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:27.505429 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:27.505429 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:27.505429 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:27.505429 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:27.505429 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:27.505429 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 06:06:27.505429 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 06:06:27.505429 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 06:06:27.505429 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 06:06:27.506422 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:27.506422 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:27.506422 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 06:06:27.506422 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:27.506422 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 06:06:27.506422 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:27.506422 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:27.506422 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:27.506422 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:27.506422 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:27.506422 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:27.506422 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:27.506422 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:27.506422 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:27.506422 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:27.506422 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:27.506422 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:27.506422 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:27.507333 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:27.507333 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:27.507333 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 06:06:27.507333 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 06:06:27.507333 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 06:06:27.507333 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 06:06:27.507333 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 06:06:27.507333 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 06:06:27.507333 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 06:06:27.507333 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:27.507333 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 257/516] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 18 06:06:27 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:27.507896 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63db764-f7f5-41ac-a64f-e18b44cab389 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:27.568124 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:27.571900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf8368-3a43-4c19-a3ca-d6052dbb6da0 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:27.572372 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:27.607719 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902d220-984a-4847-8bf6-6b24619d6414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:27.608301 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902d220-984a-4847-8bf6-6b24619d6414 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:27.608614 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902d220-984a-4847-8bf6-6b24619d6414 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:27.608895 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902d220-984a-4847-8bf6-6b24619d6414 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:27.609640 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c902d220-984a-4847-8bf6-6b24619d6414 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:27.609947 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c902d220-984a-4847-8bf6-6b24619d6414 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:27.611311 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c902d220-984a-4847-8bf6-6b24619d6414 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:27.616527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63db764-f7f5-41ac-a64f-e18b44cab389 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:27.623955 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 260/517] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:27.624788 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902d220-984a-4847-8bf6-6b24619d6414 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:27.653861 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f4137-b8f3-4253-b76f-9ee603e00d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:27.654553 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f4137-b8f3-4253-b76f-9ee603e00d76 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:27.654827 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f4137-b8f3-4253-b76f-9ee603e00d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:27.655108 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f4137-b8f3-4253-b76f-9ee603e00d76 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:27.655837 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59f4137-b8f3-4253-b76f-9ee603e00d76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:27.656149 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59f4137-b8f3-4253-b76f-9ee603e00d76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:27.657574 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59f4137-b8f3-4253-b76f-9ee603e00d76 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:27.672003 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 258/518] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:27.672987 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f4137-b8f3-4253-b76f-9ee603e00d76 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:27.730017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902d220-984a-4847-8bf6-6b24619d6414 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:27.764063 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d5adfd-1eae-4e59-8dd8-6b87d279a925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:27.764862 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5adfd-1eae-4e59-8dd8-6b87d279a925 None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:27.765220 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5adfd-1eae-4e59-8dd8-6b87d279a925 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:27.765571 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5adfd-1eae-4e59-8dd8-6b87d279a925 None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:27.766284 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d5adfd-1eae-4e59-8dd8-6b87d279a925 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:27.767722 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d5adfd-1eae-4e59-8dd8-6b87d279a925 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:27.777536 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f4137-b8f3-4253-b76f-9ee603e00d76 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:27.795624 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:27.812803 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eff481-8ee3-4b0f-a012-75265ea83a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:27.813470 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eff481-8ee3-4b0f-a012-75265ea83a26 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:27.813762 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eff481-8ee3-4b0f-a012-75265ea83a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:27.814043 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eff481-8ee3-4b0f-a012-75265ea83a26 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:27.814894 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eff481-8ee3-4b0f-a012-75265ea83a26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:27.815207 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eff481-8ee3-4b0f-a012-75265ea83a26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:27.816637 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eff481-8ee3-4b0f-a012-75265ea83a26 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:27.828296 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 259/519] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:27.829235 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eff481-8ee3-4b0f-a012-75265ea83a26 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:27.895514 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 261/520] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Jul 18 06:06:27 2022] POST /identity/v3/users => generated 247 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:06:27.896501 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d5adfd-1eae-4e59-8dd8-6b87d279a925 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:27.922746 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:27.938354 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eff481-8ee3-4b0f-a012-75265ea83a26 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:27.982606 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ad557-cc67-454d-bc73-6ad5143c8b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:27.983310 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ad557-cc67-454d-bc73-6ad5143c8b7a None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:27.983606 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ad557-cc67-454d-bc73-6ad5143c8b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:27.983886 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ad557-cc67-454d-bc73-6ad5143c8b7a None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:27.984815 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478ad557-cc67-454d-bc73-6ad5143c8b7a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:27.985134 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478ad557-cc67-454d-bc73-6ad5143c8b7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:27.986672 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478ad557-cc67-454d-bc73-6ad5143c8b7a None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:28.007082 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 260/521] 10.222.0.40 () {56 vars in 1139 bytes} [Mon Jul 18 06:06:27 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:28.007958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ad557-cc67-454d-bc73-6ad5143c8b7a None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:28.031834 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d5adfd-1eae-4e59-8dd8-6b87d279a925 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:28.049475 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:28.075286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aecf9e-f9fd-473c-a6f8-9a24635ac2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:28.076675 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aecf9e-f9fd-473c-a6f8-9a24635ac2e3 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:28.077114 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aecf9e-f9fd-473c-a6f8-9a24635ac2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:28.077585 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aecf9e-f9fd-473c-a6f8-9a24635ac2e3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:28.089271 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aecf9e-f9fd-473c-a6f8-9a24635ac2e3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:28.090733 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aecf9e-f9fd-473c-a6f8-9a24635ac2e3 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:28.109773 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-09aecf9e-f9fd-473c-a6f8-9a24635ac2e3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 18 06:06:28.109773 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:28.109773 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:28.109773 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:28.109773 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:28.109773 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:28.109773 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:28.109773 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:28.109773 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:28.109773 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:28.109773 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:28.109773 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:28.109773 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 06:06:28.109773 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 06:06:28.109773 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 06:06:28.109773 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 06:06:28.111044 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:28.111044 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:28.111044 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 06:06:28.111044 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:28.111044 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 06:06:28.111044 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:28.111044 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:28.111044 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:28.111044 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:28.111044 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:28.111044 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:28.111044 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:28.111044 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:28.111044 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:28.111044 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:28.111044 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:28.111044 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:28.111044 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:28.111961 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:28.111961 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:28.111961 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 06:06:28.111961 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 06:06:28.111961 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 06:06:28.111961 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 06:06:28.111961 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 06:06:28.111961 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 06:06:28.111961 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 18 06:06:28.111961 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:28.111961 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 262/522] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:06:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:28.111961 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aecf9e-f9fd-473c-a6f8-9a24635ac2e3 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:28.135945 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ad557-cc67-454d-bc73-6ad5143c8b7a None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:28.176383 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606be253-bf18-4d3b-95d1-331eeac74f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:28.177170 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606be253-bf18-4d3b-95d1-331eeac74f40 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:28.177461 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606be253-bf18-4d3b-95d1-331eeac74f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:28.177741 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606be253-bf18-4d3b-95d1-331eeac74f40 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:28.185443 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606be253-bf18-4d3b-95d1-331eeac74f40 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:28.187065 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606be253-bf18-4d3b-95d1-331eeac74f40 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:28.198510 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:28.203176 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-606be253-bf18-4d3b-95d1-331eeac74f40 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:28.203176 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:28.203176 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:28.203176 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:28.203176 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:28.203176 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:28.203176 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:28.203176 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:28.203176 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:28.203176 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:28.203176 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:28.203176 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:28.203176 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 06:06:28.203176 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 06:06:28.203176 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 06:06:28.204259 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 06:06:28.204259 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:28.204259 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:28.204259 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:28.204259 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:28.204259 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:28.204259 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:28.204259 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:28.204259 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:28.204259 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:28.204259 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:28.204259 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:28.204259 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:28.204259 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:28.204259 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:28.204259 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:28.204259 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:28.205716 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 06:06:28.205716 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 06:06:28.205716 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 06:06:28.205716 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 06:06:28.205716 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 06:06:28.205716 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 06:06:28.205716 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:28.205716 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:28.211093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 261/523] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:06:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:28.211093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606be253-bf18-4d3b-95d1-331eeac74f40 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:28.234150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aecf9e-f9fd-473c-a6f8-9a24635ac2e3 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:28.276808 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b0d5a2-8e50-460c-9207-2febfd92ab49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:28.277645 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b0d5a2-8e50-460c-9207-2febfd92ab49 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:28.277964 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b0d5a2-8e50-460c-9207-2febfd92ab49 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:28.278448 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b0d5a2-8e50-460c-9207-2febfd92ab49 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:28.286789 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b0d5a2-8e50-460c-9207-2febfd92ab49 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:28.288581 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b0d5a2-8e50-460c-9207-2febfd92ab49 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:28.307011 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-41b0d5a2-8e50-460c-9207-2febfd92ab49 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:28.307011 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:28.307011 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:28.307011 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:28.307011 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:28.307011 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:28.307011 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:28.307011 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:28.307011 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:28.307011 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:28.307011 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:28.307011 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:28.307011 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 06:06:28.307011 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 06:06:28.307011 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 06:06:28.307950 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 06:06:28.307950 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:28.307950 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:28.307950 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:28.307950 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:28.307950 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:28.307950 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:28.307950 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:28.307950 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:28.307950 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:28.307950 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:28.307950 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:28.307950 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:28.307950 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:28.307950 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:28.307950 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:28.307950 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:28.309044 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 06:06:28.309044 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 06:06:28.309044 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 06:06:28.309044 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 06:06:28.309044 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 06:06:28.309044 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 06:06:28.309044 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:28.309044 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:28.309044 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 263/524] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:06:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:28.309591 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b0d5a2-8e50-460c-9207-2febfd92ab49 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:28.346297 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:28.352677 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606be253-bf18-4d3b-95d1-331eeac74f40 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:28.394580 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea0f45-1e12-4750-9abf-ce62b76f8fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:28.395651 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea0f45-1e12-4750-9abf-ce62b76f8fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:28.396039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea0f45-1e12-4750-9abf-ce62b76f8fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:28.396293 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea0f45-1e12-4750-9abf-ce62b76f8fd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:28.455252 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b0d5a2-8e50-460c-9207-2febfd92ab49 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:28.493345 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b37cb-5fcb-44e7-a830-bc0a9ee4c6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:28.494621 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b37cb-5fcb-44e7-a830-bc0a9ee4c6f9 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:28.495031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b37cb-5fcb-44e7-a830-bc0a9ee4c6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:28.495595 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b37cb-5fcb-44e7-a830-bc0a9ee4c6f9 None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:28.495890 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ea0f45-1e12-4750-9abf-ce62b76f8fd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:28.496559 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b37cb-5fcb-44e7-a830-bc0a9ee4c6f9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:28.496873 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b37cb-5fcb-44e7-a830-bc0a9ee4c6f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:28.497483 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ea0f45-1e12-4750-9abf-ce62b76f8fd4 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:28.498327 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b37cb-5fcb-44e7-a830-bc0a9ee4c6f9 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:28.500436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:28.524315 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 264/525] 10.222.0.40 () {56 vars in 1137 bytes} [Mon Jul 18 06:06:28 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:28.524315 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b37cb-5fcb-44e7-a830-bc0a9ee4c6f9 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:28.606861 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:28.617890 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 262/526] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:06:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 06:06:28.617890 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea0f45-1e12-4750-9abf-ce62b76f8fd4 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:28.644003 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b37cb-5fcb-44e7-a830-bc0a9ee4c6f9 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:28.691131 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a8f63c-96d2-4cb6-8775-02330e98a781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:28.691802 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8f63c-96d2-4cb6-8775-02330e98a781 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:28.692090 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8f63c-96d2-4cb6-8775-02330e98a781 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:28.692374 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8f63c-96d2-4cb6-8775-02330e98a781 None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:28.693209 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a8f63c-96d2-4cb6-8775-02330e98a781 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:28.693529 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a8f63c-96d2-4cb6-8775-02330e98a781 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:28.694982 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a8f63c-96d2-4cb6-8775-02330e98a781 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:28.714763 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 265/527] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 18 06:06:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:28.715376 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a8f63c-96d2-4cb6-8775-02330e98a781 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:28.735782 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea0f45-1e12-4750-9abf-ce62b76f8fd4 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:28.773474 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:28.786067 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac3664-876b-4933-bac9-65f96be76f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:28.786067 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac3664-876b-4933-bac9-65f96be76f54 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:28.786067 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac3664-876b-4933-bac9-65f96be76f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:28.786067 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac3664-876b-4933-bac9-65f96be76f54 None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:28.786461 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cac3664-876b-4933-bac9-65f96be76f54 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:28.786461 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cac3664-876b-4933-bac9-65f96be76f54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:28.786461 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cac3664-876b-4933-bac9-65f96be76f54 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:28.803500 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 263/528] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 18 06:06:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:28.803500 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac3664-876b-4933-bac9-65f96be76f54 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:28.824237 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a8f63c-96d2-4cb6-8775-02330e98a781 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:28.862316 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2039353-a6c5-4745-aa7b-537bc1ddd633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:28.863292 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2039353-a6c5-4745-aa7b-537bc1ddd633 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:28.863774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2039353-a6c5-4745-aa7b-537bc1ddd633 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:28.864122 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2039353-a6c5-4745-aa7b-537bc1ddd633 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:28.871729 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:28.919904 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac3664-876b-4933-bac9-65f96be76f54 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:28.944592 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2039353-a6c5-4745-aa7b-537bc1ddd633 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:28.946080 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2039353-a6c5-4745-aa7b-537bc1ddd633 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:28.963383 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f74e1e5-fa14-4f2f-a4ca-a1399dee06e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:28.964300 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74e1e5-fa14-4f2f-a4ca-a1399dee06e5 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:28.964690 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74e1e5-fa14-4f2f-a4ca-a1399dee06e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:28.965097 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74e1e5-fa14-4f2f-a4ca-a1399dee06e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:28.976303 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f74e1e5-fa14-4f2f-a4ca-a1399dee06e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:28.977902 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f74e1e5-fa14-4f2f-a4ca-a1399dee06e5 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:28.998002 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-7f74e1e5-fa14-4f2f-a4ca-a1399dee06e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:28.998002 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:28.998002 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:28.998002 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:28.998002 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:28.998002 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:28.998002 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:28.998002 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:28.998002 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:28.998002 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:28.998002 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:28.998002 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:28.998002 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:28.998002 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:28.998002 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:28.999160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:28.999160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:28.999160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:28.999160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:28.999160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:28.999160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:28.999160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:28.999160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:28.999160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:28.999160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:28.999160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:28.999160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:28.999160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:28.999160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:28.999160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:28.999160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:28.999160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:29.005740 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:29.005740 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:29.005740 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:29.005740 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:29.005740 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 264/529] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:29.005740 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f74e1e5-fa14-4f2f-a4ca-a1399dee06e5 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:29.012318 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:29.066670 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 266/530] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:06:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 06:06:29.067665 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2039353-a6c5-4745-aa7b-537bc1ddd633 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:29.116281 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f74e1e5-fa14-4f2f-a4ca-a1399dee06e5 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:29.126244 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:29.155725 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9530434d-b57a-4ff3-a861-fcbd47c164fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:29.156398 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530434d-b57a-4ff3-a861-fcbd47c164fb None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:29.156724 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530434d-b57a-4ff3-a861-fcbd47c164fb None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:29.157069 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530434d-b57a-4ff3-a861-fcbd47c164fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:29.158067 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9530434d-b57a-4ff3-a861-fcbd47c164fb None admin] RBAC: Adding query filter params (role.id=ab10744ff77646eab23a769ab89c74aa, scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a, user.id=f92d948810ea4241810473506b654f7a) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:29.158433 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9530434d-b57a-4ff3-a861-fcbd47c164fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:29.160345 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9530434d-b57a-4ff3-a861-fcbd47c164fb None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:29.176457 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 265/531] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:06:29 2022] GET /identity/v3/role_assignments?user.id=f92d948810ea4241810473506b654f7a&scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a&role.id=ab10744ff77646eab23a769ab89c74aa => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:29.176457 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9530434d-b57a-4ff3-a861-fcbd47c164fb None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:29.203085 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2039353-a6c5-4745-aa7b-537bc1ddd633 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:29.254758 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad4927a-52b2-446d-bfea-f8eeb3843b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:29.255616 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad4927a-52b2-446d-bfea-f8eeb3843b70 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:29.255981 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad4927a-52b2-446d-bfea-f8eeb3843b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:29.256334 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad4927a-52b2-446d-bfea-f8eeb3843b70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:29.257887 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad4927a-52b2-446d-bfea-f8eeb3843b70 None admin] RBAC: Adding query filter params (role.id=ab10744ff77646eab23a769ab89c74aa, scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a, user.id=5af3a4a1bc1f4c4e9653d0a790f12606) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:29.258210 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad4927a-52b2-446d-bfea-f8eeb3843b70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:29.259767 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad4927a-52b2-446d-bfea-f8eeb3843b70 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:29.268814 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 267/532] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:06:29 2022] GET /identity/v3/role_assignments?user.id=5af3a4a1bc1f4c4e9653d0a790f12606&scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a&role.id=ab10744ff77646eab23a769ab89c74aa => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:29.269714 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad4927a-52b2-446d-bfea-f8eeb3843b70 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:29.308867 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:29.366207 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9530434d-b57a-4ff3-a861-fcbd47c164fb None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:29.421020 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad4927a-52b2-446d-bfea-f8eeb3843b70 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:29.446651 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d3043-3f16-4561-b17b-96658b2efa31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:29.447986 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d3043-3f16-4561-b17b-96658b2efa31 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:29.448268 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d3043-3f16-4561-b17b-96658b2efa31 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:29.448576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d3043-3f16-4561-b17b-96658b2efa31 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:29.449374 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d3043-3f16-4561-b17b-96658b2efa31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:29.449683 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d3043-3f16-4561-b17b-96658b2efa31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:29.451235 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d3043-3f16-4561-b17b-96658b2efa31 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:29.467163 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff085aa4-b479-410e-a361-0ffe0f458e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:29.468355 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff085aa4-b479-410e-a361-0ffe0f458e82 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:29.468871 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff085aa4-b479-410e-a361-0ffe0f458e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:29.469343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff085aa4-b479-410e-a361-0ffe0f458e82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:29.482790 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 266/533] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:29.484762 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff085aa4-b479-410e-a361-0ffe0f458e82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:29.486488 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff085aa4-b479-410e-a361-0ffe0f458e82 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:29.488172 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d3043-3f16-4561-b17b-96658b2efa31 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:29.510376 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-ff085aa4-b479-410e-a361-0ffe0f458e82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:29.510376 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:29.510376 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:29.510376 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:29.510376 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:29.510376 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:29.510376 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:29.510376 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:29.510376 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:29.510376 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:29.510376 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:29.510376 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:29.510376 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:29.510376 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:29.510376 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:29.511281 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:29.511281 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:29.511281 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:29.511281 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:29.511281 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:29.511281 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:29.511281 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:29.511281 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:29.511281 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:29.511281 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:29.511281 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:29.511281 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:29.511281 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:29.511281 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:29.511281 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:29.511281 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:29.511281 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:29.512134 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:29.512134 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:29.512134 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:29.512134 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:29.514653 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 268/534] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:29.523039 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff085aa4-b479-410e-a361-0ffe0f458e82 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:29.582020 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:29.640183 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:29.647840 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d3043-3f16-4561-b17b-96658b2efa31 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:29.683098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff085aa4-b479-410e-a361-0ffe0f458e82 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:29.696368 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c87a28-eb41-437d-bc10-e859e19da83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:29.698339 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c87a28-eb41-437d-bc10-e859e19da83b None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:29.698798 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c87a28-eb41-437d-bc10-e859e19da83b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:29.699244 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c87a28-eb41-437d-bc10-e859e19da83b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:29.706830 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c87a28-eb41-437d-bc10-e859e19da83b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:29.708573 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c87a28-eb41-437d-bc10-e859e19da83b None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:29.725493 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-02c87a28-eb41-437d-bc10-e859e19da83b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:29.725493 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:29.725493 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:29.725493 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:29.725493 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:29.725493 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:29.725493 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:29.725493 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:29.725493 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:29.725493 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:29.725493 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:29.725493 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:29.725493 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 06:06:29.725493 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 06:06:29.725493 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 06:06:29.726552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 06:06:29.726552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:29.726552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:29.726552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:29.726552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:29.726552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:29.726552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:29.726552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:29.726552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:29.726552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:29.726552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:29.726552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:29.726552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:29.726552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:29.726552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:29.726552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:29.726552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:29.727489 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 06:06:29.727489 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 06:06:29.727489 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 06:06:29.727489 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 06:06:29.727489 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 06:06:29.727489 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 06:06:29.727489 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:29.727489 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:29.729708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d87966-c19c-4a50-af9b-30b77031ff8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:29.730590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d87966-c19c-4a50-af9b-30b77031ff8b None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:29.730768 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 267/535] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:06:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:29.731075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d87966-c19c-4a50-af9b-30b77031ff8b None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:29.731480 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d87966-c19c-4a50-af9b-30b77031ff8b None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:29.732297 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d87966-c19c-4a50-af9b-30b77031ff8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:29.735106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d87966-c19c-4a50-af9b-30b77031ff8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:29.736043 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c87a28-eb41-437d-bc10-e859e19da83b None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:29.736769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d87966-c19c-4a50-af9b-30b77031ff8b None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:29.759017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 269/536] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:29.765453 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d87966-c19c-4a50-af9b-30b77031ff8b None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:29.909658 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d87966-c19c-4a50-af9b-30b77031ff8b None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:29.940316 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:29.953806 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:29.961361 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c87a28-eb41-437d-bc10-e859e19da83b None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:29.963900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfcefd7-f251-450c-9fc0-099ec2c9cfc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:29.965130 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfcefd7-f251-450c-9fc0-099ec2c9cfc5 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:29.965626 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfcefd7-f251-450c-9fc0-099ec2c9cfc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:29.966115 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfcefd7-f251-450c-9fc0-099ec2c9cfc5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:29.974607 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfcefd7-f251-450c-9fc0-099ec2c9cfc5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:29.976535 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfcefd7-f251-450c-9fc0-099ec2c9cfc5 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:29.995526 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-1bfcefd7-f251-450c-9fc0-099ec2c9cfc5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:29.995526 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:29.995526 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:29.995526 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:29.995526 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:29.995526 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:29.995526 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:29.995526 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:29.995526 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:29.995526 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:29.995526 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:29.995526 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:29.995526 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 06:06:29.995526 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 06:06:29.995526 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 06:06:29.996464 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 06:06:29.996464 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:29.996464 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:29.996464 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:29.996464 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:29.996464 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:29.996464 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:29.996464 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:29.996464 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:29.996464 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:29.996464 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:29.996464 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:29.996464 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:29.996464 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:29.996464 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:29.996464 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:29.996464 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:29.999349 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 06:06:29.999349 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 06:06:29.999349 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 06:06:29.999349 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 06:06:29.999349 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 06:06:29.999349 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 06:06:29.999349 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:29.999349 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:30.000388 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 270/537] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:06:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:06:30.012948 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfcefd7-f251-450c-9fc0-099ec2c9cfc5 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:30.034128 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e734c2-57f4-4740-b7cc-28fedd1f3cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:30.034725 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e734c2-57f4-4740-b7cc-28fedd1f3cab None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:30.035012 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e734c2-57f4-4740-b7cc-28fedd1f3cab None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:30.035294 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e734c2-57f4-4740-b7cc-28fedd1f3cab None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:30.036096 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e734c2-57f4-4740-b7cc-28fedd1f3cab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:30.036426 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e734c2-57f4-4740-b7cc-28fedd1f3cab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:30.038130 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e734c2-57f4-4740-b7cc-28fedd1f3cab None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:30.062703 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 268/538] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 18 06:06:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:06:30.067193 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e734c2-57f4-4740-b7cc-28fedd1f3cab None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:30.194800 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfcefd7-f251-450c-9fc0-099ec2c9cfc5 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:30.232284 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bf85a4-9dbf-482a-9a6b-608aa87bcddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:30.232998 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf85a4-9dbf-482a-9a6b-608aa87bcddb None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:30.233337 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf85a4-9dbf-482a-9a6b-608aa87bcddb None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:30.233626 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf85a4-9dbf-482a-9a6b-608aa87bcddb None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:30.234437 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bf85a4-9dbf-482a-9a6b-608aa87bcddb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:30.234750 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bf85a4-9dbf-482a-9a6b-608aa87bcddb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:30.236211 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bf85a4-9dbf-482a-9a6b-608aa87bcddb None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:30.236372 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e734c2-57f4-4740-b7cc-28fedd1f3cab None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:30.254279 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 271/539] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 18 06:06:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:30.255710 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9d1fd-75e1-4104-b6d2-7feda8eab211 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:30.256057 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9d1fd-75e1-4104-b6d2-7feda8eab211 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:30.256348 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9d1fd-75e1-4104-b6d2-7feda8eab211 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:30.257916 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 272/540] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:30.258626 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f9d1fd-75e1-4104-b6d2-7feda8eab211 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:30.284510 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc4a853-5a52-460b-bf44-03953d48c698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:30.285591 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4a853-5a52-460b-bf44-03953d48c698 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:30.285888 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4a853-5a52-460b-bf44-03953d48c698 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:30.286211 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4a853-5a52-460b-bf44-03953d48c698 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:30.287616 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc4a853-5a52-460b-bf44-03953d48c698 None admin] RBAC: Adding query filter params (role.id=ab10744ff77646eab23a769ab89c74aa, scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a, user.id=9824a96e2bcd4a16bfc6fe6bf4ea1ad5) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:30.288158 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc4a853-5a52-460b-bf44-03953d48c698 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:30.289427 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc4a853-5a52-460b-bf44-03953d48c698 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:30.297713 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 269/541] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:06:30 2022] GET /identity/v3/role_assignments?user.id=9824a96e2bcd4a16bfc6fe6bf4ea1ad5&scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a&role.id=ab10744ff77646eab23a769ab89c74aa => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:06:30.299619 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d72b8a-569b-4aee-93f4-305a2e0c58c9 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:30.299934 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d72b8a-569b-4aee-93f4-305a2e0c58c9 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:30.300518 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d72b8a-569b-4aee-93f4-305a2e0c58c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:30.323607 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-27d72b8a-569b-4aee-93f4-305a2e0c58c9 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:30.324912 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-27d72b8a-569b-4aee-93f4-305a2e0c58c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:30.338396 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:30.360840 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:30.410325 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f9d1fd-75e1-4104-b6d2-7feda8eab211 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:30.432745 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 270/542] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:06:30.465215 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d72b8a-569b-4aee-93f4-305a2e0c58c9 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:30.497918 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68edfd50-774e-4403-b58c-ca28d3631572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:30.498596 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68edfd50-774e-4403-b58c-ca28d3631572 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:30.498886 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68edfd50-774e-4403-b58c-ca28d3631572 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:30.499163 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68edfd50-774e-4403-b58c-ca28d3631572 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:30.500019 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68edfd50-774e-4403-b58c-ca28d3631572 None admin] RBAC: Adding query filter params (role.id=ab10744ff77646eab23a769ab89c74aa, scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a, user.id=1193330cb5644765a7abb77738929ba4) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:30.500331 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68edfd50-774e-4403-b58c-ca28d3631572 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:30.501808 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68edfd50-774e-4403-b58c-ca28d3631572 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:30.513550 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 273/543] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:06:30 2022] GET /identity/v3/role_assignments?user.id=1193330cb5644765a7abb77738929ba4&scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a&role.id=ab10744ff77646eab23a769ab89c74aa => generated 259 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:30.603969 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d72b8a-569b-4aee-93f4-305a2e0c58c9 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:30.645169 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf56203-d262-44a4-8a29-81e60ec6859a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:30.645982 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf56203-d262-44a4-8a29-81e60ec6859a None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:30.646267 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf56203-d262-44a4-8a29-81e60ec6859a None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:30.646545 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf56203-d262-44a4-8a29-81e60ec6859a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:30.659913 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf56203-d262-44a4-8a29-81e60ec6859a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:30.669093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-9bf56203-d262-44a4-8a29-81e60ec6859a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:06:30.669093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:30.669093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:30.669093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:30.669093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:30.669093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:30.669093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:30.669093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:30.669093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:30.669093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:30.669093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:30.669093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:30.669093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:30.669093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:30.669093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:30.669093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:30.670294 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:30.670294 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:30.670294 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:30.670294 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:30.670294 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:30.670294 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:30.670294 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:30.670294 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:30.670294 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:30.670294 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:30.670294 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:30.670294 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:30.670294 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:30.670294 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:30.670294 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:30.670294 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:30.671287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:30.671287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:30.671287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:30.671287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:30.671287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:30.671287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:30.671287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:30.671287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:30.671287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:30.671287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:30.671287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:06:30.671287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:30.671287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 271/544] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 18 06:06:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:06:30.677345 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68edfd50-774e-4403-b58c-ca28d3631572 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:30.834968 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68edfd50-774e-4403-b58c-ca28d3631572 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:30.852765 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:30.879951 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a614c-64a5-4244-9652-7236d1db9a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:30.880676 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a614c-64a5-4244-9652-7236d1db9a8a None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:30.880963 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a614c-64a5-4244-9652-7236d1db9a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:30.881240 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a614c-64a5-4244-9652-7236d1db9a8a None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:30.882022 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76a614c-64a5-4244-9652-7236d1db9a8a None admin] RBAC: Adding query filter params (name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:30.882329 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76a614c-64a5-4244-9652-7236d1db9a8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:30.883762 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76a614c-64a5-4244-9652-7236d1db9a8a None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:30.899334 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 274/545] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 18 06:06:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:30.904795 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf56203-d262-44a4-8a29-81e60ec6859a None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:30.915766 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:31.019777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf56203-d262-44a4-8a29-81e60ec6859a None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:31.058875 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63328b0-fd45-458b-a528-e4308f7cab49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:31.059737 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63328b0-fd45-458b-a528-e4308f7cab49 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:31.060022 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63328b0-fd45-458b-a528-e4308f7cab49 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:31.060296 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63328b0-fd45-458b-a528-e4308f7cab49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:31.073393 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63328b0-fd45-458b-a528-e4308f7cab49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:31.074912 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63328b0-fd45-458b-a528-e4308f7cab49 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:31.093047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-f63328b0-fd45-458b-a528-e4308f7cab49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:31.093047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:31.093047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:31.093047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:31.093047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:31.093047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:31.093047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:31.093047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:31.093047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:31.093047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:31.093047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:31.093047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:31.093047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:31.093047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:31.093047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:31.094042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:31.094042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:31.094042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:31.094042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:31.094042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:31.094042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:31.094042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:31.094042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:31.094042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:31.094042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:31.094042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:31.094042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:31.094042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:31.094042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:31.094042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:31.094042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:31.094042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:31.094871 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:31.094871 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:31.094871 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:31.094871 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:31.094871 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 272/546] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:31.101520 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a614c-64a5-4244-9652-7236d1db9a8a None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:31.218024 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a614c-64a5-4244-9652-7236d1db9a8a None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:31.269984 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189216db-fcd1-49bc-a3a2-09a3223fac74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:31.270663 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189216db-fcd1-49bc-a3a2-09a3223fac74 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:31.270944 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189216db-fcd1-49bc-a3a2-09a3223fac74 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:31.271215 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189216db-fcd1-49bc-a3a2-09a3223fac74 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:31.271943 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189216db-fcd1-49bc-a3a2-09a3223fac74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:31.272247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189216db-fcd1-49bc-a3a2-09a3223fac74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:31.273715 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189216db-fcd1-49bc-a3a2-09a3223fac74 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:31.287317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 275/547] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:31.293308 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63328b0-fd45-458b-a528-e4308f7cab49 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:31.390560 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:31.454154 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63328b0-fd45-458b-a528-e4308f7cab49 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:31.509847 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae7a6e-a450-4c61-814c-f7e86c0201bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:31.510666 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae7a6e-a450-4c61-814c-f7e86c0201bc None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:31.510956 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae7a6e-a450-4c61-814c-f7e86c0201bc None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:31.511236 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae7a6e-a450-4c61-814c-f7e86c0201bc None admin] PATH_INFO: `/v3/users/swift` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:31.524370 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cae7a6e-a450-4c61-814c-f7e86c0201bc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:31.526177 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cae7a6e-a450-4c61-814c-f7e86c0201bc None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:31.548104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-1cae7a6e-a450-4c61-814c-f7e86c0201bc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 06:06:31.548104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:31.548104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:31.548104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:31.548104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:31.548104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:31.548104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:31.548104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:31.548104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:31.548104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:31.548104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:31.548104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:31.548104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 06:06:31.548104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 06:06:31.548104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 06:06:31.548104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 06:06:31.550448 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:31.550448 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:31.550448 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 06:06:31.550448 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:31.550448 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 06:06:31.550448 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:31.550448 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:31.550448 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:31.550448 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:31.550448 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:31.550448 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:31.550448 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:31.550448 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:31.550448 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:31.550448 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:31.550448 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:31.550448 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:31.550448 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:31.552729 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:31.552729 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:31.552729 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 06:06:31.552729 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 06:06:31.552729 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 06:06:31.552729 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 06:06:31.552729 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 06:06:31.552729 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 06:06:31.552729 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 06:06:31.552729 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:31.552729 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 273/548] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 18 06:06:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:31.556948 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189216db-fcd1-49bc-a3a2-09a3223fac74 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:31.639757 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:31.767035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189216db-fcd1-49bc-a3a2-09a3223fac74 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:31.821729 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c04c2-5c44-48c8-84a8-b60990c43ec6 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:31.822049 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c04c2-5c44-48c8-84a8-b60990c43ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:31.822317 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c04c2-5c44-48c8-84a8-b60990c43ec6 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:31.823348 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5beb6c-6e24-4a2a-ba99-4601ae566fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:31.824052 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 274/549] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:06:31.824681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5beb6c-6e24-4a2a-ba99-4601ae566fe1 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:31.825158 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5beb6c-6e24-4a2a-ba99-4601ae566fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:31.825598 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5beb6c-6e24-4a2a-ba99-4601ae566fe1 None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:31.826707 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5beb6c-6e24-4a2a-ba99-4601ae566fe1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:31.827335 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5beb6c-6e24-4a2a-ba99-4601ae566fe1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:31.829014 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5beb6c-6e24-4a2a-ba99-4601ae566fe1 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:31.836815 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac80b7e-7f95-44b5-bf6f-898e70287c2c None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:31.837105 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac80b7e-7f95-44b5-bf6f-898e70287c2c None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:31.837389 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac80b7e-7f95-44b5-bf6f-898e70287c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:31.859040 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 276/550] 10.222.0.40 () {56 vars in 1135 bytes} [Mon Jul 18 06:06:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:31.860798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cbcb0f-eb98-49b2-b834-4a7adae24d27 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:31.864318 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cbcb0f-eb98-49b2-b834-4a7adae24d27 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:31.864780 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cbcb0f-eb98-49b2-b834-4a7adae24d27 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:31.865593 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-2ac80b7e-7f95-44b5-bf6f-898e70287c2c None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:31.866675 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 277/551] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:31.867180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-2ac80b7e-7f95-44b5-bf6f-898e70287c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:31.872561 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cbcb0f-eb98-49b2-b834-4a7adae24d27 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:31.960664 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 275/552] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 06:06:31.962344 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883e5c5-8ba9-46a9-9d94-82996c3b2cd0 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:31.962833 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883e5c5-8ba9-46a9-9d94-82996c3b2cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:31.963267 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883e5c5-8ba9-46a9-9d94-82996c3b2cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:31.986336 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:32.001875 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-6883e5c5-8ba9-46a9-9d94-82996c3b2cd0 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:32.003125 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-6883e5c5-8ba9-46a9-9d94-82996c3b2cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:32.007383 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cbcb0f-eb98-49b2-b834-4a7adae24d27 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:32.045649 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6753cc19-1762-4f02-831f-aa0af8ec14c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:32.046264 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753cc19-1762-4f02-831f-aa0af8ec14c5 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:32.046549 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753cc19-1762-4f02-831f-aa0af8ec14c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:32.046857 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753cc19-1762-4f02-831f-aa0af8ec14c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:32.061006 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:32.088808 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 276/553] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:06:32.090323 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883e5c5-8ba9-46a9-9d94-82996c3b2cd0 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:32.118614 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6753cc19-1762-4f02-831f-aa0af8ec14c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:32.120217 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6753cc19-1762-4f02-831f-aa0af8ec14c5 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:32.206393 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883e5c5-8ba9-46a9-9d94-82996c3b2cd0 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:32.215399 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:32.224966 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 278/554] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:06:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 06:06:32.225978 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6753cc19-1762-4f02-831f-aa0af8ec14c5 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:32.248102 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29c5932-dc49-4d29-acb5-b3015aceac54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:32.249332 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c5932-dc49-4d29-acb5-b3015aceac54 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:32.249858 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c5932-dc49-4d29-acb5-b3015aceac54 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:32.250371 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c5932-dc49-4d29-acb5-b3015aceac54 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:32.263212 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29c5932-dc49-4d29-acb5-b3015aceac54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:32.271915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-c29c5932-dc49-4d29-acb5-b3015aceac54 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:06:32.271915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:32.271915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:32.271915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:32.271915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:32.271915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:32.271915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:32.271915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:32.271915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:32.271915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:32.271915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:32.271915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:32.271915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:32.271915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:32.271915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:32.271915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:32.272884 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:32.272884 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:32.272884 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:32.272884 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:32.272884 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:32.272884 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:32.272884 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:32.272884 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:32.272884 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:32.272884 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:32.272884 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:32.272884 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:32.272884 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:32.272884 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:32.272884 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:32.272884 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:32.273754 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:32.273754 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:32.273754 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:32.273754 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:32.273754 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:32.273754 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:32.273754 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:32.273754 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:32.273754 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:32.273754 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:32.273754 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:06:32.273754 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:32.281653 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 277/555] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 18 06:06:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:32.282892 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29c5932-dc49-4d29-acb5-b3015aceac54 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:32.329162 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:32.376468 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6753cc19-1762-4f02-831f-aa0af8ec14c5 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:32.398855 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29c5932-dc49-4d29-acb5-b3015aceac54 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:32.439474 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6f88b-5640-4ba1-b935-ce4d781aa2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:32.439848 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c3c763-0c1c-4747-96ec-fd97b3a3e358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:32.440963 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6f88b-5640-4ba1-b935-ce4d781aa2d5 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:32.441197 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3c763-0c1c-4747-96ec-fd97b3a3e358 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:32.441688 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3c763-0c1c-4747-96ec-fd97b3a3e358 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:32.441795 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6f88b-5640-4ba1-b935-ce4d781aa2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:32.442178 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3c763-0c1c-4747-96ec-fd97b3a3e358 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:32.442475 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6f88b-5640-4ba1-b935-ce4d781aa2d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:32.456778 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb6f88b-5640-4ba1-b935-ce4d781aa2d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:32.459119 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb6f88b-5640-4ba1-b935-ce4d781aa2d5 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:32.464006 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c3c763-0c1c-4747-96ec-fd97b3a3e358 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:32.479962 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-5cb6f88b-5640-4ba1-b935-ce4d781aa2d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:32.479962 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:32.479962 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:32.479962 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:32.479962 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:32.479962 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:32.479962 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:32.479962 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:32.479962 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:32.479962 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:32.479962 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:32.479962 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:32.479962 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:32.479962 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:32.479962 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:32.486288 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:32.486288 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:32.486288 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:32.486288 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:32.486288 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:32.486288 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:32.486288 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:32.486288 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:32.486288 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:32.486288 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:32.486288 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:32.486288 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:32.486288 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:32.486288 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:32.486288 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:32.486288 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:32.486288 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:32.487154 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:32.487154 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:32.487154 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:32.487154 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:32.487379 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-14c3c763-0c1c-4747-96ec-fd97b3a3e358 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:06:32.487379 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:32.487379 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:32.487379 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:32.487379 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:32.487379 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:32.487379 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:32.487379 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:32.487379 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:32.487379 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:32.487379 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:32.487379 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:32.487379 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:32.487379 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:32.487379 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:32.487379 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:32.488265 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:32.488265 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:32.488265 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:32.488265 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:32.488265 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:32.488265 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:32.488265 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:32.488265 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:32.488265 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:32.488265 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:32.488265 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:32.488265 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:32.488265 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:32.488265 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:32.488265 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:32.488265 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:32.490205 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:32.490205 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:32.490205 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:32.490205 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:32.490205 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:32.490205 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:32.490205 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:32.490205 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:32.490205 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:32.490205 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:32.490205 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:06:32.490205 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:32.490843 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 278/556] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:32.490843 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6f88b-5640-4ba1-b935-ce4d781aa2d5 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:32.490978 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 279/557] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 18 06:06:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:32.490978 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c3c763-0c1c-4747-96ec-fd97b3a3e358 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:32.585948 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:32.595372 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:32.606701 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6f88b-5640-4ba1-b935-ce4d781aa2d5 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:32.645667 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef67d6b-0b2e-4229-91c1-3a3b15c2e3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:32.646782 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef67d6b-0b2e-4229-91c1-3a3b15c2e3be None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:32.647265 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef67d6b-0b2e-4229-91c1-3a3b15c2e3be None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:32.647726 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef67d6b-0b2e-4229-91c1-3a3b15c2e3be None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:32.648706 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef67d6b-0b2e-4229-91c1-3a3b15c2e3be None admin] RBAC: Adding query filter params (name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:32.649218 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef67d6b-0b2e-4229-91c1-3a3b15c2e3be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:32.652392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef67d6b-0b2e-4229-91c1-3a3b15c2e3be None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:32.662455 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 279/558] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 18 06:06:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:32.663668 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef67d6b-0b2e-4229-91c1-3a3b15c2e3be None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:32.671333 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c3c763-0c1c-4747-96ec-fd97b3a3e358 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:32.731188 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b50d9ab-2684-4a5c-bdcc-24ed2008deac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:32.732286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50d9ab-2684-4a5c-bdcc-24ed2008deac None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:32.732875 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50d9ab-2684-4a5c-bdcc-24ed2008deac None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:32.733353 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50d9ab-2684-4a5c-bdcc-24ed2008deac None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:32.734636 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b50d9ab-2684-4a5c-bdcc-24ed2008deac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:32.735150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b50d9ab-2684-4a5c-bdcc-24ed2008deac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:32.737544 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b50d9ab-2684-4a5c-bdcc-24ed2008deac None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:32.758468 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 280/559] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:32.760545 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b50d9ab-2684-4a5c-bdcc-24ed2008deac None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:32.780794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef67d6b-0b2e-4229-91c1-3a3b15c2e3be None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:32.820940 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e7371-3735-48c6-b8e4-d408b4f6749d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:32.821708 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e7371-3735-48c6-b8e4-d408b4f6749d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:32.822619 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e7371-3735-48c6-b8e4-d408b4f6749d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:32.822899 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e7371-3735-48c6-b8e4-d408b4f6749d None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:32.823672 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e7371-3735-48c6-b8e4-d408b4f6749d None admin] RBAC: Adding query filter params (name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:32.823987 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e7371-3735-48c6-b8e4-d408b4f6749d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:32.825525 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e7371-3735-48c6-b8e4-d408b4f6749d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:32.839666 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 280/560] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 18 06:06:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:32.839666 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e7371-3735-48c6-b8e4-d408b4f6749d None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:32.870879 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:32.952016 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b50d9ab-2684-4a5c-bdcc-24ed2008deac None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:32.975708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:33.003544 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e7371-3735-48c6-b8e4-d408b4f6749d None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:33.016096 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deca362-e694-4d13-b31e-26e526d774b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:33.016994 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deca362-e694-4d13-b31e-26e526d774b5 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:33.017574 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deca362-e694-4d13-b31e-26e526d774b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:33.018018 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deca362-e694-4d13-b31e-26e526d774b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:33.029173 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deca362-e694-4d13-b31e-26e526d774b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:33.030627 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deca362-e694-4d13-b31e-26e526d774b5 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:33.042570 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099fabbd-b027-4416-a9fd-6963c38d28a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:33.043369 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099fabbd-b027-4416-a9fd-6963c38d28a9 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:33.043703 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099fabbd-b027-4416-a9fd-6963c38d28a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:33.043981 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099fabbd-b027-4416-a9fd-6963c38d28a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:33.050132 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-3deca362-e694-4d13-b31e-26e526d774b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:33.050132 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:33.050132 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:33.050132 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:33.050132 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:33.050132 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:33.050132 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:33.050132 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:33.050132 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:33.050132 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:33.050132 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:33.050132 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:33.050132 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:33.050132 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:33.050132 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:33.051047 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:33.051047 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:33.051047 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:33.051047 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:33.051047 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:33.051047 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:33.051047 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:33.051047 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:33.051047 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:33.051047 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:33.051047 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:33.051047 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:33.051047 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:33.051047 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:33.051047 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:33.051047 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:33.051047 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:33.051943 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:33.051943 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:33.051943 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:33.051943 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:33.052179 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099fabbd-b027-4416-a9fd-6963c38d28a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:33.052251 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 281/561] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:33.052251 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deca362-e694-4d13-b31e-26e526d774b5 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:33.052978 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099fabbd-b027-4416-a9fd-6963c38d28a9 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:33.068837 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-099fabbd-b027-4416-a9fd-6963c38d28a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:33.068837 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:33.068837 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:33.068837 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:33.068837 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:33.068837 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:33.068837 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:33.068837 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:33.068837 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:33.068837 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:33.068837 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:33.068837 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:33.068837 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 06:06:33.068837 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 06:06:33.068837 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 06:06:33.070024 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 06:06:33.070024 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:33.070024 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:33.070024 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:33.070024 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:33.070024 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:33.070024 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:33.070024 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:33.070024 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:33.070024 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:33.070024 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:33.070024 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:33.070024 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:33.070024 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:33.070024 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:33.070024 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:33.070024 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:33.071163 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 06:06:33.071163 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 06:06:33.071163 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 06:06:33.071163 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 06:06:33.071163 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 06:06:33.071163 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 06:06:33.071163 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:33.071163 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:33.071163 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 281/562] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:06:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:33.072443 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82217331-5ad2-4d90-aa7a-d6c1472d21c2 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:33.072443 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82217331-5ad2-4d90-aa7a-d6c1472d21c2 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:33.072585 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82217331-5ad2-4d90-aa7a-d6c1472d21c2 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:33.074413 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 282/563] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:33.075312 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94168f8a-48b6-4ade-9077-351f93f946c6 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:33.075683 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94168f8a-48b6-4ade-9077-351f93f946c6 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:33.075866 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94168f8a-48b6-4ade-9077-351f93f946c6 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:33.077094 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 283/564] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:33.077781 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94168f8a-48b6-4ade-9077-351f93f946c6 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:33.166491 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:33.168451 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deca362-e694-4d13-b31e-26e526d774b5 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:33.194305 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94168f8a-48b6-4ade-9077-351f93f946c6 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:33.204932 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8560166d-7bd7-48b1-a173-5a21ac00fa8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:33.205719 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8560166d-7bd7-48b1-a173-5a21ac00fa8b None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:33.206007 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8560166d-7bd7-48b1-a173-5a21ac00fa8b None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:33.206294 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8560166d-7bd7-48b1-a173-5a21ac00fa8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:33.217070 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8560166d-7bd7-48b1-a173-5a21ac00fa8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:33.218545 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8560166d-7bd7-48b1-a173-5a21ac00fa8b None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:33.230164 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca33fc3-123d-437d-8761-556390401ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:33.230829 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca33fc3-123d-437d-8761-556390401ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:33.231121 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca33fc3-123d-437d-8761-556390401ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:33.231398 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca33fc3-123d-437d-8761-556390401ea0 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:33.232117 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca33fc3-123d-437d-8761-556390401ea0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:33.232439 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca33fc3-123d-437d-8761-556390401ea0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:33.233818 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca33fc3-123d-437d-8761-556390401ea0 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:33.236866 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-8560166d-7bd7-48b1-a173-5a21ac00fa8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:33.236866 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:33.236866 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:33.236866 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:33.236866 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:33.236866 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:33.236866 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:33.236866 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:33.236866 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:33.236866 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:33.236866 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:33.236866 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:33.236866 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:33.236866 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:33.236866 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:33.237798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:33.237798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:33.237798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:33.237798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:33.237798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:33.237798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:33.237798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:33.237798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:33.237798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:33.237798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:33.237798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:33.237798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:33.237798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:33.237798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:33.237798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:33.237798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:33.237798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:33.238709 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:33.238709 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:33.238709 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:33.238709 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:33.238709 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 282/565] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:33.238987 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8560166d-7bd7-48b1-a173-5a21ac00fa8b None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:33.248458 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 284/566] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:33.248458 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912fa7d-65ee-46f3-a225-630a3192bc86 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:33.248458 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912fa7d-65ee-46f3-a225-630a3192bc86 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:33.248458 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912fa7d-65ee-46f3-a225-630a3192bc86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:33.261913 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:33.265233 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-9912fa7d-65ee-46f3-a225-630a3192bc86 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:33.266452 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-9912fa7d-65ee-46f3-a225-630a3192bc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:33.337252 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 285/567] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:06:33.339138 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021b007-1781-40f1-ac87-0aaeb21be81e None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:33.339428 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021b007-1781-40f1-ac87-0aaeb21be81e None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:33.339734 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021b007-1781-40f1-ac87-0aaeb21be81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:33.356515 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8560166d-7bd7-48b1-a173-5a21ac00fa8b None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:33.357907 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-5021b007-1781-40f1-ac87-0aaeb21be81e None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:33.359862 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-5021b007-1781-40f1-ac87-0aaeb21be81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:33.391740 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e2a077-4703-4e51-9b2d-73dc1929eb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:33.392380 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e2a077-4703-4e51-9b2d-73dc1929eb5f None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:33.392704 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e2a077-4703-4e51-9b2d-73dc1929eb5f None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:33.392987 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e2a077-4703-4e51-9b2d-73dc1929eb5f None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:33.393779 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e2a077-4703-4e51-9b2d-73dc1929eb5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:33.394178 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e2a077-4703-4e51-9b2d-73dc1929eb5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:33.395656 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e2a077-4703-4e51-9b2d-73dc1929eb5f None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:33.408527 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 283/568] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 18 06:06:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:33.409372 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e2a077-4703-4e51-9b2d-73dc1929eb5f None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:33.422294 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:33.430608 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 286/569] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:06:33.432091 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5021b007-1781-40f1-ac87-0aaeb21be81e None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:33.497236 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:33.519861 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e2a077-4703-4e51-9b2d-73dc1929eb5f None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:33.540011 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5021b007-1781-40f1-ac87-0aaeb21be81e None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:33.555362 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe049ec-c002-4154-8262-4e22abc2eb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:33.556021 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe049ec-c002-4154-8262-4e22abc2eb5f None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:33.556334 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe049ec-c002-4154-8262-4e22abc2eb5f None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:33.556825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe049ec-c002-4154-8262-4e22abc2eb5f None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:33.557726 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe049ec-c002-4154-8262-4e22abc2eb5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:33.558040 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe049ec-c002-4154-8262-4e22abc2eb5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:33.559401 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe049ec-c002-4154-8262-4e22abc2eb5f None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:33.572843 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 284/570] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:33.573970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe049ec-c002-4154-8262-4e22abc2eb5f None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:33.579196 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e793a8-f6f9-4378-8333-309eff4fa7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:33.580017 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e793a8-f6f9-4378-8333-309eff4fa7e7 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:33.580301 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e793a8-f6f9-4378-8333-309eff4fa7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:33.580610 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e793a8-f6f9-4378-8333-309eff4fa7e7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:33.588973 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e793a8-f6f9-4378-8333-309eff4fa7e7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:33.590509 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e793a8-f6f9-4378-8333-309eff4fa7e7 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:33.605876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-72e793a8-f6f9-4378-8333-309eff4fa7e7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 06:06:33.605876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:33.605876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:33.605876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:33.605876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:33.605876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:33.605876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:33.605876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:33.605876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:33.605876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:33.605876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:33.605876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:33.605876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 06:06:33.605876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 06:06:33.605876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 06:06:33.605876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 06:06:33.607047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:33.607047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:33.607047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 06:06:33.607047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:33.607047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 06:06:33.607047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:33.607047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:33.607047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:33.607047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:33.607047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:33.607047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:33.607047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:33.607047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:33.607047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:33.607047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:33.607047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:33.607047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:33.607047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:33.608240 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:33.608240 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:33.608240 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 06:06:33.608240 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 06:06:33.608240 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 06:06:33.608240 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 06:06:33.608240 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 06:06:33.608240 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 06:06:33.608240 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 06:06:33.608240 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:33.608240 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 287/571] 10.222.0.40 () {56 vars in 1093 bytes} [Mon Jul 18 06:06:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:33.608240 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e793a8-f6f9-4378-8333-309eff4fa7e7 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:33.666982 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:33.687763 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe049ec-c002-4154-8262-4e22abc2eb5f None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:33.713258 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e793a8-f6f9-4378-8333-309eff4fa7e7 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:33.723248 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae41bbe-fca0-41a2-8df3-f05d3a591c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:33.724044 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae41bbe-fca0-41a2-8df3-f05d3a591c53 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:33.724331 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae41bbe-fca0-41a2-8df3-f05d3a591c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:33.724647 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae41bbe-fca0-41a2-8df3-f05d3a591c53 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:33.727400 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:33.736004 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae41bbe-fca0-41a2-8df3-f05d3a591c53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:33.744283 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-4ae41bbe-fca0-41a2-8df3-f05d3a591c53 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:06:33.744283 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:33.744283 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:33.744283 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:33.744283 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:33.744283 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:33.744283 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:33.744283 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:33.744283 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:33.744283 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:33.744283 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:33.744283 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:33.744283 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:33.744283 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:33.744283 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:33.744283 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:33.745741 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:33.745741 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:33.745741 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:33.745741 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:33.745741 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:33.745741 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:33.745741 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:33.745741 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:33.745741 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:33.745741 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:33.745741 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:33.745741 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:33.745741 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:33.745741 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:33.745741 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:33.745741 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:33.746809 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:33.746809 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:33.746809 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:33.746809 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:33.746809 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:33.746809 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:33.746809 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:33.746809 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:33.746809 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:33.746809 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:33.746809 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:06:33.746809 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:33.746809 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 285/572] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 18 06:06:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:33.746809 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae41bbe-fca0-41a2-8df3-f05d3a591c53 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:33.753140 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32dd92d-96df-45d4-8416-f6e8a3977246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:33.753804 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32dd92d-96df-45d4-8416-f6e8a3977246 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:33.754090 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32dd92d-96df-45d4-8416-f6e8a3977246 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:33.754368 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32dd92d-96df-45d4-8416-f6e8a3977246 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:33.755212 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32dd92d-96df-45d4-8416-f6e8a3977246 None admin] RBAC: Adding query filter params (role.id=ab10744ff77646eab23a769ab89c74aa, scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a, user.id=0681f5fbea6d415c9ad2f3afae256a5c) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:33.755525 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32dd92d-96df-45d4-8416-f6e8a3977246 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:33.757012 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32dd92d-96df-45d4-8416-f6e8a3977246 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:33.763021 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 288/573] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:06:33 2022] GET /identity/v3/role_assignments?user.id=0681f5fbea6d415c9ad2f3afae256a5c&scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a&role.id=ab10744ff77646eab23a769ab89c74aa => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:33.763819 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32dd92d-96df-45d4-8416-f6e8a3977246 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:33.880150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae41bbe-fca0-41a2-8df3-f05d3a591c53 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:33.888335 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32dd92d-96df-45d4-8416-f6e8a3977246 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:33.910851 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:33.921430 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14957485-e643-4697-ab27-c1ba43e5b48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:33.922239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14957485-e643-4697-ab27-c1ba43e5b48f None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:33.922534 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14957485-e643-4697-ab27-c1ba43e5b48f None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:33.922815 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14957485-e643-4697-ab27-c1ba43e5b48f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:33.933090 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb67999e-215f-406a-89ad-c4bdd12e3c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:33.933930 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb67999e-215f-406a-89ad-c4bdd12e3c0f None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:33.934221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb67999e-215f-406a-89ad-c4bdd12e3c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:33.934401 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14957485-e643-4697-ab27-c1ba43e5b48f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:33.934500 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb67999e-215f-406a-89ad-c4bdd12e3c0f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:33.942265 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb67999e-215f-406a-89ad-c4bdd12e3c0f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:33.942417 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-14957485-e643-4697-ab27-c1ba43e5b48f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:06:33.942417 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:33.942417 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:33.942417 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:33.942417 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:33.942417 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:33.942417 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:33.942417 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:33.942417 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:33.942417 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:33.942417 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:33.942417 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:33.942417 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:33.942417 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:33.942417 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:33.942417 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:33.943436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:33.943436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:33.943436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:33.943436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:33.943436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:33.943436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:33.943436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:33.943436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:33.943436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:33.943436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:33.943436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:33.943436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:33.943436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:33.943436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:33.943436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:33.943436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:33.944319 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:33.944319 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:33.944319 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:33.944319 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:33.944319 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:33.944319 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:33.944319 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:33.944319 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:33.944319 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:33.944319 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:33.944319 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:06:33.944319 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:33.944319 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 286/574] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 18 06:06:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:33.945881 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14957485-e643-4697-ab27-c1ba43e5b48f None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:33.948643 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb67999e-215f-406a-89ad-c4bdd12e3c0f None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:33.965801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-cb67999e-215f-406a-89ad-c4bdd12e3c0f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 06:06:33.965801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:33.965801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:33.965801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:33.965801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:33.965801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:33.965801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:33.965801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:33.965801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:33.965801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:33.965801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:33.965801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:33.965801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 06:06:33.965801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 06:06:33.965801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 06:06:33.965801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 06:06:33.966844 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:33.966844 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:33.966844 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 06:06:33.966844 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:33.966844 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 06:06:33.966844 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:33.966844 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:33.966844 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:33.966844 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:33.966844 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:33.966844 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:33.966844 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:33.966844 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:33.966844 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:33.966844 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:33.966844 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:33.966844 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:33.966844 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:33.967791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:33.967791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:33.967791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 06:06:33.967791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 06:06:33.967791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 06:06:33.967791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 06:06:33.967791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 06:06:33.967791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 06:06:33.967791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 06:06:33.967791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:33.967791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 289/575] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:06:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:33.972331 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb67999e-215f-406a-89ad-c4bdd12e3c0f None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:34.012047 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:34.060555 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14957485-e643-4697-ab27-c1ba43e5b48f None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:34.085067 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb67999e-215f-406a-89ad-c4bdd12e3c0f None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:34.098448 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d4716-583f-4fb9-8aed-4d133d178cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:34.099133 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d4716-583f-4fb9-8aed-4d133d178cc3 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:34.099418 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d4716-583f-4fb9-8aed-4d133d178cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:34.099698 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d4716-583f-4fb9-8aed-4d133d178cc3 None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:34.100584 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4d4716-583f-4fb9-8aed-4d133d178cc3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:34.100897 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4d4716-583f-4fb9-8aed-4d133d178cc3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:34.102383 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4d4716-583f-4fb9-8aed-4d133d178cc3 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:34.126559 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 287/576] 10.222.0.40 () {56 vars in 1133 bytes} [Mon Jul 18 06:06:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:34.126748 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf6a6ef-9d0a-4aef-bc01-d0c7968e7310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:34.126845 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d4716-583f-4fb9-8aed-4d133d178cc3 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:34.126950 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf6a6ef-9d0a-4aef-bc01-d0c7968e7310 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:34.126950 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf6a6ef-9d0a-4aef-bc01-d0c7968e7310 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:34.126950 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf6a6ef-9d0a-4aef-bc01-d0c7968e7310 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:34.126950 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf6a6ef-9d0a-4aef-bc01-d0c7968e7310 None admin] RBAC: Adding query filter params (name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:34.126950 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf6a6ef-9d0a-4aef-bc01-d0c7968e7310 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:34.126950 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf6a6ef-9d0a-4aef-bc01-d0c7968e7310 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:34.139887 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 290/577] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 18 06:06:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:34.139887 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf6a6ef-9d0a-4aef-bc01-d0c7968e7310 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:34.183946 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:34.241344 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d4716-583f-4fb9-8aed-4d133d178cc3 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:34.248359 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf6a6ef-9d0a-4aef-bc01-d0c7968e7310 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:34.256685 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:34.289047 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65282e8c-187a-4915-bc03-63816473b23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:34.289816 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65282e8c-187a-4915-bc03-63816473b23f None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:34.290104 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65282e8c-187a-4915-bc03-63816473b23f None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:34.290384 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65282e8c-187a-4915-bc03-63816473b23f None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:34.292011 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65282e8c-187a-4915-bc03-63816473b23f None admin] RBAC: Adding query filter params (name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:34.292336 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65282e8c-187a-4915-bc03-63816473b23f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:34.293756 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65282e8c-187a-4915-bc03-63816473b23f None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:34.295391 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b627d86-6983-4f62-948a-400858ece159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:34.296147 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b627d86-6983-4f62-948a-400858ece159 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:34.296402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b627d86-6983-4f62-948a-400858ece159 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:34.296712 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b627d86-6983-4f62-948a-400858ece159 None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:34.297621 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b627d86-6983-4f62-948a-400858ece159 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:34.297937 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b627d86-6983-4f62-948a-400858ece159 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:34.299387 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b627d86-6983-4f62-948a-400858ece159 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:34.303231 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 288/578] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 18 06:06:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:34.309013 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65282e8c-187a-4915-bc03-63816473b23f None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:34.319791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 291/579] 10.222.0.40 () {56 vars in 1137 bytes} [Mon Jul 18 06:06:34 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:34.320625 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b627d86-6983-4f62-948a-400858ece159 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:34.434203 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65282e8c-187a-4915-bc03-63816473b23f None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:34.441050 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:34.444969 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b627d86-6983-4f62-948a-400858ece159 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:34.474352 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41f0575-4898-4b17-a081-e0b40a0d6cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:34.475494 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f0575-4898-4b17-a081-e0b40a0d6cf2 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:34.475781 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f0575-4898-4b17-a081-e0b40a0d6cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:34.476061 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f0575-4898-4b17-a081-e0b40a0d6cf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:34.482641 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8323cb9-ec54-41ca-956b-cf46c052d9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:34.483713 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8323cb9-ec54-41ca-956b-cf46c052d9e6 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:34.483713 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8323cb9-ec54-41ca-956b-cf46c052d9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:34.483996 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8323cb9-ec54-41ca-956b-cf46c052d9e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:34.488764 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41f0575-4898-4b17-a081-e0b40a0d6cf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:34.490556 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41f0575-4898-4b17-a081-e0b40a0d6cf2 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:34.495263 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8323cb9-ec54-41ca-956b-cf46c052d9e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:34.496706 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8323cb9-ec54-41ca-956b-cf46c052d9e6 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:34.511117 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-d41f0575-4898-4b17-a081-e0b40a0d6cf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:34.511117 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:34.511117 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:34.511117 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:34.511117 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:34.511117 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:34.511117 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:34.511117 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:34.511117 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:34.511117 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:34.511117 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:34.511117 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:34.511117 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:34.511117 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:34.511117 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:34.512080 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:34.512080 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:34.512080 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:34.512080 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:34.512080 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:34.512080 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:34.512080 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:34.512080 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:34.512080 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:34.512080 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:34.512080 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:34.512080 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:34.512080 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:34.512080 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:34.512080 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:34.512080 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:34.512080 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:34.513032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:34.513032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:34.513032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:34.513032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:34.519109 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 289/580] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:34.519109 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41f0575-4898-4b17-a081-e0b40a0d6cf2 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:34.519322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-a8323cb9-ec54-41ca-956b-cf46c052d9e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:34.519322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:34.519322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:34.519322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:34.519322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:34.519322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:34.519322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:34.519322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:34.519322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:34.519322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:34.519322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:34.519322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:34.519322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:34.519322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:34.519322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:34.520221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:34.520221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:34.520221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:34.520221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:34.520221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:34.520221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:34.520221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:34.520221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:34.520221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:34.520221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:34.520221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:34.520221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:34.520221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:34.520221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:34.520221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:34.520221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:34.520221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:34.521126 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:34.521126 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:34.521126 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:34.521126 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:34.521126 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 292/581] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:34.521126 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8323cb9-ec54-41ca-956b-cf46c052d9e6 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:34.554810 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:34.631839 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8323cb9-ec54-41ca-956b-cf46c052d9e6 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:34.634160 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41f0575-4898-4b17-a081-e0b40a0d6cf2 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:34.673610 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de6cd0-4ae3-46cf-b0dd-6d8042d21eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:34.674371 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c011dc-56d5-4417-bf3d-2a5a6890bfb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:34.674495 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de6cd0-4ae3-46cf-b0dd-6d8042d21eeb None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:34.674684 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de6cd0-4ae3-46cf-b0dd-6d8042d21eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:34.674964 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de6cd0-4ae3-46cf-b0dd-6d8042d21eeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:34.676024 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c011dc-56d5-4417-bf3d-2a5a6890bfb0 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:34.676528 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c011dc-56d5-4417-bf3d-2a5a6890bfb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:34.677005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c011dc-56d5-4417-bf3d-2a5a6890bfb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:34.690843 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de6cd0-4ae3-46cf-b0dd-6d8042d21eeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:34.690983 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c011dc-56d5-4417-bf3d-2a5a6890bfb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:34.692331 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de6cd0-4ae3-46cf-b0dd-6d8042d21eeb None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:34.692481 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c011dc-56d5-4417-bf3d-2a5a6890bfb0 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:34.711193 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-48c011dc-56d5-4417-bf3d-2a5a6890bfb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:34.711193 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:34.711193 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:34.711193 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:34.711193 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:34.711193 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:34.711193 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:34.711193 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:34.711193 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:34.711193 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:34.711193 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:34.711193 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:34.711193 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:34.711193 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:34.711193 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:34.712211 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:34.712211 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:34.712211 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:34.712211 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:34.712211 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:34.712211 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:34.712211 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:34.712211 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:34.712211 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:34.712211 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:34.712211 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:34.712211 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:34.712211 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:34.712211 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:34.712211 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:34.712211 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:34.712211 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:34.713252 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:34.713252 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:34.713252 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:34.713252 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:34.713479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-28de6cd0-4ae3-46cf-b0dd-6d8042d21eeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:34.713479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:34.713479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:34.713479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:34.713479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:34.713479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:34.713479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:34.713479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:34.713479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:34.713479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:34.713479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:34.713479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:34.713479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:34.713479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:34.713479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:34.714313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:34.714313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:34.714313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:34.714313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:34.714313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:34.714313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:34.714313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:34.714313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:34.714313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:34.714313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:34.714313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:34.714313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:34.714313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:34.714313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:34.714313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:34.714313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:34.714313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:34.716585 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:34.716585 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:34.716585 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:34.716585 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:34.716990 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 290/582] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:34.716990 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c011dc-56d5-4417-bf3d-2a5a6890bfb0 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:34.718420 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 293/583] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:34.719071 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de6cd0-4ae3-46cf-b0dd-6d8042d21eeb None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:34.722342 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:34.809827 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:34.829446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c011dc-56d5-4417-bf3d-2a5a6890bfb0 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:34.835999 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de6cd0-4ae3-46cf-b0dd-6d8042d21eeb None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:34.868872 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4405c5fd-157a-4803-a4d6-bf516021625e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:34.869612 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405c5fd-157a-4803-a4d6-bf516021625e None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:34.869922 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405c5fd-157a-4803-a4d6-bf516021625e None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:34.870182 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405c5fd-157a-4803-a4d6-bf516021625e None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:34.870915 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4405c5fd-157a-4803-a4d6-bf516021625e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:34.871227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4405c5fd-157a-4803-a4d6-bf516021625e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:34.872730 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4405c5fd-157a-4803-a4d6-bf516021625e None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:34.875085 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f63928b-1d90-4d69-a453-bb2d42e9b42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:34.875767 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63928b-1d90-4d69-a453-bb2d42e9b42d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:34.876053 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63928b-1d90-4d69-a453-bb2d42e9b42d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:34.876333 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63928b-1d90-4d69-a453-bb2d42e9b42d None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:34.877141 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f63928b-1d90-4d69-a453-bb2d42e9b42d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:34.877453 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f63928b-1d90-4d69-a453-bb2d42e9b42d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:34.878901 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f63928b-1d90-4d69-a453-bb2d42e9b42d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:34.890442 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 291/584] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:34.890442 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4405c5fd-157a-4803-a4d6-bf516021625e None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:34.906038 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 294/585] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:34.906038 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f63928b-1d90-4d69-a453-bb2d42e9b42d None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:34.994046 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:35.021820 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4405c5fd-157a-4803-a4d6-bf516021625e None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:35.039672 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f63928b-1d90-4d69-a453-bb2d42e9b42d None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:35.062798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af985c0-e5d1-4526-af81-853719ceefad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:35.063393 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af985c0-e5d1-4526-af81-853719ceefad None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:35.063691 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af985c0-e5d1-4526-af81-853719ceefad None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:35.063973 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af985c0-e5d1-4526-af81-853719ceefad None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:35.064786 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af985c0-e5d1-4526-af81-853719ceefad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:35.065098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af985c0-e5d1-4526-af81-853719ceefad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:35.066534 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af985c0-e5d1-4526-af81-853719ceefad None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:35.078338 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:35.082568 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 292/586] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:35.083490 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af985c0-e5d1-4526-af81-853719ceefad None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:35.093939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d657e28-f413-4235-a6e1-7d0ba72f6580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:35.094633 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d657e28-f413-4235-a6e1-7d0ba72f6580 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:35.098421 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d657e28-f413-4235-a6e1-7d0ba72f6580 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:35.098421 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d657e28-f413-4235-a6e1-7d0ba72f6580 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:35.098421 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d657e28-f413-4235-a6e1-7d0ba72f6580 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:35.098421 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d657e28-f413-4235-a6e1-7d0ba72f6580 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:35.098421 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d657e28-f413-4235-a6e1-7d0ba72f6580 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:35.111287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 295/587] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:35.112163 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d657e28-f413-4235-a6e1-7d0ba72f6580 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:35.211081 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af985c0-e5d1-4526-af81-853719ceefad None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:35.231390 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d657e28-f413-4235-a6e1-7d0ba72f6580 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:35.255730 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:35.256732 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d39dc7a-739e-4928-bb5e-d78d89d69f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:35.257547 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39dc7a-739e-4928-bb5e-d78d89d69f32 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:35.257830 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39dc7a-739e-4928-bb5e-d78d89d69f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:35.258109 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39dc7a-739e-4928-bb5e-d78d89d69f32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:35.267247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d39dc7a-739e-4928-bb5e-d78d89d69f32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:35.268531 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d39dc7a-739e-4928-bb5e-d78d89d69f32 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:35.270019 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d05fa85-1960-4912-b16f-10a106692bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:35.270581 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05fa85-1960-4912-b16f-10a106692bc6 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:35.270873 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05fa85-1960-4912-b16f-10a106692bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:35.271250 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05fa85-1960-4912-b16f-10a106692bc6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:35.280973 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d05fa85-1960-4912-b16f-10a106692bc6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:35.282468 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d05fa85-1960-4912-b16f-10a106692bc6 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:35.285098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-5d39dc7a-739e-4928-bb5e-d78d89d69f32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:35.285098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:35.285098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:35.285098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:35.285098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:35.285098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:35.285098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:35.285098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:35.285098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:35.285098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:35.285098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:35.285098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:35.285098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 06:06:35.285098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 06:06:35.285098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 06:06:35.286041 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 06:06:35.286041 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:35.286041 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:35.286041 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:35.286041 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:35.286041 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:35.286041 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:35.286041 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:35.286041 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:35.286041 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:35.286041 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:35.286041 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:35.286041 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:35.286041 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:35.286041 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:35.286041 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:35.286041 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:35.287243 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 06:06:35.287243 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 06:06:35.287243 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 06:06:35.287243 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 06:06:35.287243 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 06:06:35.287243 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 06:06:35.287243 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:35.287243 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:35.287243 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 293/588] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:06:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:35.287835 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d39dc7a-739e-4928-bb5e-d78d89d69f32 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:35.304098 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-3d05fa85-1960-4912-b16f-10a106692bc6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 06:06:35.304098 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:35.304098 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:35.304098 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:35.304098 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:35.304098 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:35.304098 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:35.304098 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:35.304098 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:35.304098 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:35.304098 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:35.304098 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:35.304098 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 06:06:35.304098 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 06:06:35.304098 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 06:06:35.304098 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 06:06:35.305356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:35.305356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:35.305356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 06:06:35.305356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:35.305356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 06:06:35.305356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:35.305356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:35.305356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:35.305356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:35.305356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:35.305356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:35.305356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:35.305356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:35.305356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:35.305356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:35.305356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:35.305356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:35.305356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:35.306607 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:35.306607 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:35.306607 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 06:06:35.306607 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 06:06:35.306607 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 06:06:35.306607 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 06:06:35.306607 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 06:06:35.306607 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 06:06:35.306607 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 06:06:35.306607 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:35.312201 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 296/589] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 18 06:06:35 2022] GET /identity/v3/users/neutron => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:35.312201 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d05fa85-1960-4912-b16f-10a106692bc6 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:35.364851 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:35.413084 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d39dc7a-739e-4928-bb5e-d78d89d69f32 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:35.423890 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d05fa85-1960-4912-b16f-10a106692bc6 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:35.455930 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6e6c52-f3b1-443a-be8e-00b3ce8536d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:35.456762 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e6c52-f3b1-443a-be8e-00b3ce8536d1 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:35.457049 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e6c52-f3b1-443a-be8e-00b3ce8536d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:35.457324 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e6c52-f3b1-443a-be8e-00b3ce8536d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:35.464835 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6e6c52-f3b1-443a-be8e-00b3ce8536d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:35.466283 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6e6c52-f3b1-443a-be8e-00b3ce8536d1 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:35.468440 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23817ed-ca3f-4812-8df3-b5806046c30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:35.469250 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23817ed-ca3f-4812-8df3-b5806046c30b None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:35.469542 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23817ed-ca3f-4812-8df3-b5806046c30b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:35.469826 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23817ed-ca3f-4812-8df3-b5806046c30b None admin] PATH_INFO: `/v3/users/glance` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:35.478137 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23817ed-ca3f-4812-8df3-b5806046c30b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:35.479850 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23817ed-ca3f-4812-8df3-b5806046c30b None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:35.481439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-2b6e6c52-f3b1-443a-be8e-00b3ce8536d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:35.481439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:35.481439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:35.481439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:35.481439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:35.481439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:35.481439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:35.481439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:35.481439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:35.481439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:35.481439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:35.481439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:35.481439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 06:06:35.481439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 06:06:35.481439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 06:06:35.482371 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 06:06:35.482371 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:35.482371 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:35.482371 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:35.482371 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:35.482371 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:35.482371 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:35.482371 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:35.482371 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:35.482371 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:35.482371 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:35.482371 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:35.482371 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:35.482371 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:35.482371 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:35.482371 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:35.482371 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:35.483275 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 06:06:35.483275 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 06:06:35.483275 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 06:06:35.483275 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 06:06:35.483275 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 06:06:35.483275 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 06:06:35.483275 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:35.483275 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:35.483275 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 294/590] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:06:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:35.484352 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6e6c52-f3b1-443a-be8e-00b3ce8536d1 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:35.501262 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-c23817ed-ca3f-4812-8df3-b5806046c30b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 18 06:06:35.501262 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:35.501262 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:35.501262 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:35.501262 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:35.501262 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:35.501262 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:35.501262 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:35.501262 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:35.501262 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:35.501262 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:35.501262 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:35.501262 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 06:06:35.501262 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 06:06:35.501262 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 06:06:35.501262 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 06:06:35.502276 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:35.502276 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:35.502276 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 06:06:35.502276 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:35.502276 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 06:06:35.502276 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:35.502276 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:35.502276 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:35.502276 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:35.502276 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:35.502276 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:35.502276 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:35.502276 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:35.502276 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:35.502276 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:35.502276 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:35.502276 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:35.502276 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:35.503880 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:35.503880 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:35.503880 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 06:06:35.503880 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 06:06:35.503880 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 06:06:35.503880 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 06:06:35.503880 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 06:06:35.503880 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 06:06:35.503880 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 18 06:06:35.503880 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:35.503880 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 297/591] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:06:35 2022] GET /identity/v3/users/glance => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:35.503880 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23817ed-ca3f-4812-8df3-b5806046c30b None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:35.549780 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:35.603784 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6e6c52-f3b1-443a-be8e-00b3ce8536d1 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:35.612593 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23817ed-ca3f-4812-8df3-b5806046c30b None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:35.632083 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:35.647531 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e1827-8dff-4a17-9efe-21d05228163f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:35.648245 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e1827-8dff-4a17-9efe-21d05228163f None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:35.648568 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e1827-8dff-4a17-9efe-21d05228163f None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:35.648850 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e1827-8dff-4a17-9efe-21d05228163f None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:35.649690 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7e1827-8dff-4a17-9efe-21d05228163f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:35.650524 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7e1827-8dff-4a17-9efe-21d05228163f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:35.652137 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7e1827-8dff-4a17-9efe-21d05228163f None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:35.654700 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8748a-bdb5-4f2b-8698-b70978dd30d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:35.656613 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8748a-bdb5-4f2b-8698-b70978dd30d9 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:35.657096 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8748a-bdb5-4f2b-8698-b70978dd30d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:35.657557 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8748a-bdb5-4f2b-8698-b70978dd30d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:35.658551 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a8748a-bdb5-4f2b-8698-b70978dd30d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:35.659049 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a8748a-bdb5-4f2b-8698-b70978dd30d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:35.661135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a8748a-bdb5-4f2b-8698-b70978dd30d9 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:35.672531 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 298/592] 10.222.0.40 () {56 vars in 1139 bytes} [Mon Jul 18 06:06:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:35.673380 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e1827-8dff-4a17-9efe-21d05228163f None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:35.691050 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 295/593] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 18 06:06:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:35.691050 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8748a-bdb5-4f2b-8698-b70978dd30d9 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:35.811666 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:35.815274 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e1827-8dff-4a17-9efe-21d05228163f None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:35.819915 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8748a-bdb5-4f2b-8698-b70978dd30d9 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:35.855243 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e513e3a-19fb-46cd-949b-012e5f732316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:35.855913 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e513e3a-19fb-46cd-949b-012e5f732316 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:35.856196 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e513e3a-19fb-46cd-949b-012e5f732316 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:35.856399 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4040ff3-6c43-4e24-a57c-3b894b80c05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:35.856544 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e513e3a-19fb-46cd-949b-012e5f732316 None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:35.857103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4040ff3-6c43-4e24-a57c-3b894b80c05f None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:35.857303 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e513e3a-19fb-46cd-949b-012e5f732316 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:35.857411 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4040ff3-6c43-4e24-a57c-3b894b80c05f None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:35.857613 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e513e3a-19fb-46cd-949b-012e5f732316 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:35.857704 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4040ff3-6c43-4e24-a57c-3b894b80c05f None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:35.858484 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4040ff3-6c43-4e24-a57c-3b894b80c05f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:35.858799 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4040ff3-6c43-4e24-a57c-3b894b80c05f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:35.859049 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e513e3a-19fb-46cd-949b-012e5f732316 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:35.860219 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4040ff3-6c43-4e24-a57c-3b894b80c05f None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:35.871929 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 299/594] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 18 06:06:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:35.873141 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e513e3a-19fb-46cd-949b-012e5f732316 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:35.879728 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 296/595] 10.222.0.40 () {56 vars in 1137 bytes} [Mon Jul 18 06:06:35 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:35.880338 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4040ff3-6c43-4e24-a57c-3b894b80c05f None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:35.920695 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:35.982001 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4040ff3-6c43-4e24-a57c-3b894b80c05f None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:35.982477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e513e3a-19fb-46cd-949b-012e5f732316 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:36.018117 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b04d3d8-9b41-41de-a13f-d1e397f9367d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:36.018383 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7debe7c-ed30-4b2b-91e3-037b0797388f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:36.018930 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04d3d8-9b41-41de-a13f-d1e397f9367d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:36.019195 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7debe7c-ed30-4b2b-91e3-037b0797388f None admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:36.019287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04d3d8-9b41-41de-a13f-d1e397f9367d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:36.019481 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7debe7c-ed30-4b2b-91e3-037b0797388f None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:36.019583 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04d3d8-9b41-41de-a13f-d1e397f9367d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:36.019757 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7debe7c-ed30-4b2b-91e3-037b0797388f None admin] PATH_INFO: `/v3/projects/8ed50a8a3e184bde9269d7b0d9573d4a/users/f92d948810ea4241810473506b654f7a/roles/ab10744ff77646eab23a769ab89c74aa` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:36.033587 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b04d3d8-9b41-41de-a13f-d1e397f9367d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:36.035097 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b04d3d8-9b41-41de-a13f-d1e397f9367d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:36.035855 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7debe7c-ed30-4b2b-91e3-037b0797388f None admin] RBAC: Authorizing `identity:create_grant(project_id=8ed50a8a3e184bde9269d7b0d9573d4a, user_id=f92d948810ea4241810473506b654f7a, role_id=ab10744ff77646eab23a769ab89c74aa)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:36.037603 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7debe7c-ed30-4b2b-91e3-037b0797388f None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:36.053527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-4b04d3d8-9b41-41de-a13f-d1e397f9367d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:36.053527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:36.053527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:36.053527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:36.053527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:36.053527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:36.053527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:36.053527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:36.053527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:36.053527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:36.053527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:36.053527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:36.053527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:36.053527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:36.053527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:36.054595 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:36.054595 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:36.054595 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:36.054595 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:36.054595 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:36.054595 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:36.054595 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:36.054595 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:36.054595 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:36.054595 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:36.054595 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:36.054595 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:36.054595 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:36.054595 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:36.054595 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:36.054595 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:36.054595 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:36.055624 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:36.055624 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:36.055624 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:36.055624 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:36.055624 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 300/596] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:36.055944 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb90288-db06-416b-a89d-e28a920cb50f None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:36.056801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb90288-db06-416b-a89d-e28a920cb50f None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:36.057086 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb90288-db06-416b-a89d-e28a920cb50f None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:36.058358 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 301/597] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:36.059044 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb90288-db06-416b-a89d-e28a920cb50f None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:36.075162 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:36.113202 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 297/598] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 18 06:06:35 2022] PUT /identity/v3/projects/8ed50a8a3e184bde9269d7b0d9573d4a/users/f92d948810ea4241810473506b654f7a/roles/ab10744ff77646eab23a769ab89c74aa => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:06:36.114560 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7debe7c-ed30-4b2b-91e3-037b0797388f None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:36.203027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb90288-db06-416b-a89d-e28a920cb50f None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:36.248485 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa443cbf-cf60-4644-bd53-d62f2c64d48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:36.249400 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa443cbf-cf60-4644-bd53-d62f2c64d48d None admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:36.249688 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa443cbf-cf60-4644-bd53-d62f2c64d48d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:36.249971 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa443cbf-cf60-4644-bd53-d62f2c64d48d None admin] PATH_INFO: `/v3/projects/8ed50a8a3e184bde9269d7b0d9573d4a/users/5af3a4a1bc1f4c4e9653d0a790f12606/roles/ab10744ff77646eab23a769ab89c74aa` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:36.254521 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:36.267794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa443cbf-cf60-4644-bd53-d62f2c64d48d None admin] RBAC: Authorizing `identity:create_grant(project_id=8ed50a8a3e184bde9269d7b0d9573d4a, user_id=5af3a4a1bc1f4c4e9653d0a790f12606, role_id=ab10744ff77646eab23a769ab89c74aa)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:36.268002 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7debe7c-ed30-4b2b-91e3-037b0797388f None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:36.269928 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa443cbf-cf60-4644-bd53-d62f2c64d48d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:36.310538 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0997cb46-62ee-4771-abc2-99c9b289f2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:36.311385 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997cb46-62ee-4771-abc2-99c9b289f2dc None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:36.311665 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997cb46-62ee-4771-abc2-99c9b289f2dc None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:36.311941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997cb46-62ee-4771-abc2-99c9b289f2dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:36.322073 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0997cb46-62ee-4771-abc2-99c9b289f2dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:36.323561 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0997cb46-62ee-4771-abc2-99c9b289f2dc None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:36.341941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-0997cb46-62ee-4771-abc2-99c9b289f2dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:36.341941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:36.341941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:36.341941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:36.341941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:36.341941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:36.341941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:36.341941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:36.341941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:36.341941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:36.341941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:36.341941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:36.341941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:36.341941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:36.341941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:36.343061 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:36.343061 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:36.343061 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:36.343061 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:36.343061 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:36.343061 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:36.343061 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:36.343061 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:36.343061 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:36.343061 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:36.343061 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:36.343061 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:36.343061 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:36.343061 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:36.343061 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:36.343061 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:36.343061 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:36.343988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:36.343988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:36.343988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:36.343988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:36.343988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 298/599] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:36.344407 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0997cb46-62ee-4771-abc2-99c9b289f2dc None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:36.400442 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 302/600] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 18 06:06:36 2022] PUT /identity/v3/projects/8ed50a8a3e184bde9269d7b0d9573d4a/users/5af3a4a1bc1f4c4e9653d0a790f12606/roles/ab10744ff77646eab23a769ab89c74aa => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:06:36.400442 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b6cc2-0a73-46bf-acae-28b329df7f45 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:36.400442 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b6cc2-0a73-46bf-acae-28b329df7f45 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:36.400442 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b6cc2-0a73-46bf-acae-28b329df7f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:36.416256 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-4c0b6cc2-0a73-46bf-acae-28b329df7f45 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:36.418068 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4c0b6cc2-0a73-46bf-acae-28b329df7f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:36.484167 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0997cb46-62ee-4771-abc2-99c9b289f2dc None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:36.511182 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:36.527717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4107dec2-b8c4-493b-b0e6-b2b7dbfe2943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:36.528562 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107dec2-b8c4-493b-b0e6-b2b7dbfe2943 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:36.528975 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107dec2-b8c4-493b-b0e6-b2b7dbfe2943 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:36.529323 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107dec2-b8c4-493b-b0e6-b2b7dbfe2943 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:36.530155 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4107dec2-b8c4-493b-b0e6-b2b7dbfe2943 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:36.530471 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4107dec2-b8c4-493b-b0e6-b2b7dbfe2943 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:36.531940 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4107dec2-b8c4-493b-b0e6-b2b7dbfe2943 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:36.532954 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 303/601] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:06:36.534331 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b6cc2-0a73-46bf-acae-28b329df7f45 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:36.547511 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:36.562787 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 299/602] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:36.563668 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4107dec2-b8c4-493b-b0e6-b2b7dbfe2943 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:36.663580 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b6cc2-0a73-46bf-acae-28b329df7f45 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:36.693798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4107dec2-b8c4-493b-b0e6-b2b7dbfe2943 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:36.705772 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e9f83-5e46-40fd-8019-edac0334b2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:36.706875 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e9f83-5e46-40fd-8019-edac0334b2f4 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:36.707373 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e9f83-5e46-40fd-8019-edac0334b2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:36.707870 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e9f83-5e46-40fd-8019-edac0334b2f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:36.708891 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9e9f83-5e46-40fd-8019-edac0334b2f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:36.709442 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9e9f83-5e46-40fd-8019-edac0334b2f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:36.711094 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9e9f83-5e46-40fd-8019-edac0334b2f4 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:36.725273 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 304/603] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:36.726530 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e9f83-5e46-40fd-8019-edac0334b2f4 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:36.732969 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800f5cf-b3f0-4bf5-82e7-5688914ead24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:36.733779 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800f5cf-b3f0-4bf5-82e7-5688914ead24 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:36.734067 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800f5cf-b3f0-4bf5-82e7-5688914ead24 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:36.734580 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800f5cf-b3f0-4bf5-82e7-5688914ead24 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:36.745973 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800f5cf-b3f0-4bf5-82e7-5688914ead24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:36.754087 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-2800f5cf-b3f0-4bf5-82e7-5688914ead24 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:06:36.754087 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:36.754087 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:36.754087 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:36.754087 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:36.754087 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:36.754087 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:36.754087 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:36.754087 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:36.754087 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:36.754087 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:36.754087 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:36.754087 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:36.754087 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:36.754087 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:36.754087 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:36.755171 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:36.755171 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:36.755171 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:36.755171 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:36.755171 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:36.755171 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:36.755171 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:36.755171 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:36.755171 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:36.755171 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:36.755171 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:36.755171 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:36.755171 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:36.755171 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:36.755171 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:36.755171 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:36.756126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:36.756126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:36.756126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:36.756126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:36.756126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:36.756126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:36.756126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:36.756126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:36.756126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:36.756126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:36.756126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:06:36.756126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:36.756126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 300/604] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 18 06:06:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:36.758006 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800f5cf-b3f0-4bf5-82e7-5688914ead24 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:36.792913 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:36.835546 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:36.841852 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e9f83-5e46-40fd-8019-edac0334b2f4 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:36.937497 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c05a2a-7c2f-451c-a85b-d24b43e64204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:36.938305 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c05a2a-7c2f-451c-a85b-d24b43e64204 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:36.938594 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c05a2a-7c2f-451c-a85b-d24b43e64204 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:36.938701 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800f5cf-b3f0-4bf5-82e7-5688914ead24 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:36.938876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c05a2a-7c2f-451c-a85b-d24b43e64204 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:36.946351 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c05a2a-7c2f-451c-a85b-d24b43e64204 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:36.948563 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c05a2a-7c2f-451c-a85b-d24b43e64204 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:36.964477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-91c05a2a-7c2f-451c-a85b-d24b43e64204 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:36.964477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:36.964477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:36.964477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:36.964477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:36.964477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:36.964477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:36.964477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:36.964477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:36.964477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:36.964477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:36.964477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:36.964477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 06:06:36.964477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 06:06:36.964477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 06:06:36.965424 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 06:06:36.965424 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:36.965424 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:36.965424 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:36.965424 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:36.965424 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:36.965424 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:36.965424 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:36.965424 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:36.965424 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:36.965424 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:36.965424 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:36.965424 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:36.965424 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:36.965424 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:36.965424 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:36.965424 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:36.966339 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 06:06:36.966339 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 06:06:36.966339 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 06:06:36.966339 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 06:06:36.966339 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 06:06:36.966339 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 06:06:36.966339 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:36.966339 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:36.966339 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 305/605] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:06:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:36.966832 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c05a2a-7c2f-451c-a85b-d24b43e64204 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:36.975366 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fee6b23-b27b-437e-b522-151fc0973fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:36.976174 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee6b23-b27b-437e-b522-151fc0973fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:36.976499 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee6b23-b27b-437e-b522-151fc0973fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:36.976974 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee6b23-b27b-437e-b522-151fc0973fc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:36.984853 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fee6b23-b27b-437e-b522-151fc0973fc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:36.986237 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fee6b23-b27b-437e-b522-151fc0973fc4 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:37.001818 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-5fee6b23-b27b-437e-b522-151fc0973fc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:37.001818 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:37.001818 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:37.001818 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:37.001818 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:37.001818 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:37.001818 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:37.001818 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:37.001818 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:37.001818 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:37.001818 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:37.001818 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:37.001818 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 06:06:37.001818 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 06:06:37.001818 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 06:06:37.004004 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 06:06:37.004004 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:37.004004 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:37.004004 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:37.004004 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:37.004004 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:37.004004 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:37.004004 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:37.004004 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:37.004004 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:37.004004 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:37.004004 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:37.004004 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:37.004004 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:37.004004 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:37.004004 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:37.004004 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:37.005531 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 06:06:37.005531 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 06:06:37.005531 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 06:06:37.005531 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 06:06:37.005531 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 06:06:37.005531 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 06:06:37.005531 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:37.005531 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:37.005531 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 301/606] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:06:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:37.005531 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fee6b23-b27b-437e-b522-151fc0973fc4 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:37.083999 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c05a2a-7c2f-451c-a85b-d24b43e64204 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:37.101137 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:37.116331 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fee6b23-b27b-437e-b522-151fc0973fc4 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:37.138436 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f3f7d5-68f8-4303-9f1a-48d347445636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:37.139268 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3f7d5-68f8-4303-9f1a-48d347445636 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:37.139652 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3f7d5-68f8-4303-9f1a-48d347445636 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:37.140026 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3f7d5-68f8-4303-9f1a-48d347445636 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:37.140924 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f3f7d5-68f8-4303-9f1a-48d347445636 None admin] RBAC: Adding query filter params (name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:37.143715 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f3f7d5-68f8-4303-9f1a-48d347445636 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:37.144354 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:37.150249 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f3f7d5-68f8-4303-9f1a-48d347445636 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:37.159879 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 306/607] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 18 06:06:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:37.162274 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f3f7d5-68f8-4303-9f1a-48d347445636 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:37.165381 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8435671-7ecc-43c5-b09c-67c18ee4fb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:37.166702 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8435671-7ecc-43c5-b09c-67c18ee4fb30 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:37.166990 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8435671-7ecc-43c5-b09c-67c18ee4fb30 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:37.167332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8435671-7ecc-43c5-b09c-67c18ee4fb30 None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:37.168331 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8435671-7ecc-43c5-b09c-67c18ee4fb30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:37.168667 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8435671-7ecc-43c5-b09c-67c18ee4fb30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:37.170120 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8435671-7ecc-43c5-b09c-67c18ee4fb30 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:37.185481 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 302/608] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 18 06:06:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:37.186534 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8435671-7ecc-43c5-b09c-67c18ee4fb30 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:37.270429 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f3f7d5-68f8-4303-9f1a-48d347445636 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:37.308070 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8435671-7ecc-43c5-b09c-67c18ee4fb30 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:37.309195 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df680cd-3632-468b-9035-291284644bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:37.310120 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df680cd-3632-468b-9035-291284644bf1 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:37.310408 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df680cd-3632-468b-9035-291284644bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:37.310685 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df680cd-3632-468b-9035-291284644bf1 None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:37.312536 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df680cd-3632-468b-9035-291284644bf1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:37.312856 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df680cd-3632-468b-9035-291284644bf1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:37.314343 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df680cd-3632-468b-9035-291284644bf1 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:37.331551 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 307/609] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 18 06:06:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:37.332963 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df680cd-3632-468b-9035-291284644bf1 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:37.347656 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e58d1-8a34-4183-98b0-2a523df17e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:37.348562 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e58d1-8a34-4183-98b0-2a523df17e31 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:37.348855 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e58d1-8a34-4183-98b0-2a523df17e31 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:37.349130 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e58d1-8a34-4183-98b0-2a523df17e31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:37.361649 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e58d1-8a34-4183-98b0-2a523df17e31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:37.363535 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e58d1-8a34-4183-98b0-2a523df17e31 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:37.374689 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:37.384189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-6e0e58d1-8a34-4183-98b0-2a523df17e31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:37.384189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:37.384189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:37.384189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:37.384189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:37.384189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:37.384189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:37.384189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:37.384189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:37.384189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:37.384189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:37.384189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:37.384189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:37.384189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:37.384189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:37.385508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:37.385508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:37.385508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:37.385508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:37.385508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:37.385508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:37.385508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:37.385508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:37.385508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:37.385508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:37.385508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:37.385508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:37.385508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:37.385508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:37.385508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:37.385508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:37.385508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:37.386617 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:37.386617 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:37.386617 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:37.386617 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:37.389425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 303/610] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:37.396998 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e58d1-8a34-4183-98b0-2a523df17e31 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:37.437043 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:37.465196 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df680cd-3632-468b-9035-291284644bf1 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:37.511498 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff935dbc-5c65-4a4b-ade8-a1971d84cde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:37.512797 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff935dbc-5c65-4a4b-ade8-a1971d84cde9 None admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:37.513093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff935dbc-5c65-4a4b-ade8-a1971d84cde9 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:37.513360 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff935dbc-5c65-4a4b-ade8-a1971d84cde9 None admin] PATH_INFO: `/v3/projects/8ed50a8a3e184bde9269d7b0d9573d4a/users/9824a96e2bcd4a16bfc6fe6bf4ea1ad5/roles/ab10744ff77646eab23a769ab89c74aa` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:37.517852 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e58d1-8a34-4183-98b0-2a523df17e31 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:37.533592 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff935dbc-5c65-4a4b-ade8-a1971d84cde9 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ed50a8a3e184bde9269d7b0d9573d4a, user_id=9824a96e2bcd4a16bfc6fe6bf4ea1ad5, role_id=ab10744ff77646eab23a769ab89c74aa)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:37.535281 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff935dbc-5c65-4a4b-ade8-a1971d84cde9 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:37.557173 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e6ca37-d614-4e5b-840b-0dcf3f1ff922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:37.558338 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e6ca37-d614-4e5b-840b-0dcf3f1ff922 None admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:37.558689 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e6ca37-d614-4e5b-840b-0dcf3f1ff922 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:37.559032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e6ca37-d614-4e5b-840b-0dcf3f1ff922 None admin] PATH_INFO: `/v3/projects/8ed50a8a3e184bde9269d7b0d9573d4a/users/1193330cb5644765a7abb77738929ba4/roles/ab10744ff77646eab23a769ab89c74aa` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:37.575242 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e6ca37-d614-4e5b-840b-0dcf3f1ff922 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ed50a8a3e184bde9269d7b0d9573d4a, user_id=1193330cb5644765a7abb77738929ba4, role_id=ab10744ff77646eab23a769ab89c74aa)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:37.576819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e6ca37-d614-4e5b-840b-0dcf3f1ff922 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:37.644393 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 308/611] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 18 06:06:37 2022] PUT /identity/v3/projects/8ed50a8a3e184bde9269d7b0d9573d4a/users/9824a96e2bcd4a16bfc6fe6bf4ea1ad5/roles/ab10744ff77646eab23a769ab89c74aa => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:06:37.645610 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff935dbc-5c65-4a4b-ade8-a1971d84cde9 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:37.676883 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 304/612] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 18 06:06:37 2022] PUT /identity/v3/projects/8ed50a8a3e184bde9269d7b0d9573d4a/users/1193330cb5644765a7abb77738929ba4/roles/ab10744ff77646eab23a769ab89c74aa => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:06:37.804131 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:37.825241 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff935dbc-5c65-4a4b-ade8-a1971d84cde9 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:37.877482 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31752afa-f341-41b2-a38b-0cb9ee758145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:37.878180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31752afa-f341-41b2-a38b-0cb9ee758145 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:37.878465 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31752afa-f341-41b2-a38b-0cb9ee758145 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:37.878746 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31752afa-f341-41b2-a38b-0cb9ee758145 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:37.879490 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31752afa-f341-41b2-a38b-0cb9ee758145 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:37.879802 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31752afa-f341-41b2-a38b-0cb9ee758145 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:37.881244 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31752afa-f341-41b2-a38b-0cb9ee758145 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:37.895610 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 309/613] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:37.902880 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e6ca37-d614-4e5b-840b-0dcf3f1ff922 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:37.993285 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:38.014331 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e6ca37-d614-4e5b-840b-0dcf3f1ff922 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:38.051335 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbed504-945e-4e1c-aaba-1f2f0e6324cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:38.052210 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbed504-945e-4e1c-aaba-1f2f0e6324cb None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:38.052526 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbed504-945e-4e1c-aaba-1f2f0e6324cb None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:38.052804 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbed504-945e-4e1c-aaba-1f2f0e6324cb None admin] PATH_INFO: `/v3/users/swift` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:38.063471 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbed504-945e-4e1c-aaba-1f2f0e6324cb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:38.064889 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbed504-945e-4e1c-aaba-1f2f0e6324cb None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:38.082149 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-6cbed504-945e-4e1c-aaba-1f2f0e6324cb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 06:06:38.082149 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:38.082149 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:38.082149 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:38.082149 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:38.082149 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:38.082149 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:38.082149 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:38.082149 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:38.082149 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:38.082149 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:38.082149 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:38.082149 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 06:06:38.082149 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 06:06:38.082149 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 06:06:38.082149 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 06:06:38.083343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:38.083343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:38.083343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 06:06:38.083343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:38.083343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 06:06:38.083343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:38.083343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:38.083343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:38.083343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:38.083343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:38.083343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:38.083343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:38.083343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:38.083343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:38.083343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:38.083343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:38.083343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:38.083343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:38.084848 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:38.084848 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:38.084848 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 06:06:38.084848 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 06:06:38.084848 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 06:06:38.084848 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 06:06:38.084848 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 06:06:38.084848 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 06:06:38.084848 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 06:06:38.084848 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:38.084848 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 305/614] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 18 06:06:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:38.092887 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31752afa-f341-41b2-a38b-0cb9ee758145 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:38.246276 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31752afa-f341-41b2-a38b-0cb9ee758145 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:38.299032 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c79581-9bc9-4371-a71c-5e4b8a2df148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:38.299730 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c79581-9bc9-4371-a71c-5e4b8a2df148 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:38.300027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c79581-9bc9-4371-a71c-5e4b8a2df148 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:38.300308 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c79581-9bc9-4371-a71c-5e4b8a2df148 None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:38.301202 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c79581-9bc9-4371-a71c-5e4b8a2df148 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:38.301516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c79581-9bc9-4371-a71c-5e4b8a2df148 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:38.302979 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c79581-9bc9-4371-a71c-5e4b8a2df148 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:38.324889 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 310/615] 10.222.0.40 () {56 vars in 1135 bytes} [Mon Jul 18 06:06:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:38.331137 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbed504-945e-4e1c-aaba-1f2f0e6324cb None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:38.520039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1b44f-4bf9-4a1d-b341-e39bb1153fd1 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:38.520821 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1b44f-4bf9-4a1d-b341-e39bb1153fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:38.520821 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1b44f-4bf9-4a1d-b341-e39bb1153fd1 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:38.522290 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 311/616] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:38.527325 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbed504-945e-4e1c-aaba-1f2f0e6324cb None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:38.528144 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62eb676-745c-42e9-b41d-71b2ddada705 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:38.528492 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62eb676-745c-42e9-b41d-71b2ddada705 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:38.528786 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62eb676-745c-42e9-b41d-71b2ddada705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:38.540781 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:38.557644 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-b62eb676-745c-42e9-b41d-71b2ddada705 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:38.558883 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-b62eb676-745c-42e9-b41d-71b2ddada705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:38.585574 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11822a72-2502-4f51-8923-c252892ab762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:38.586311 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11822a72-2502-4f51-8923-c252892ab762 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:38.586604 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11822a72-2502-4f51-8923-c252892ab762 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:38.586888 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11822a72-2502-4f51-8923-c252892ab762 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:38.598821 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11822a72-2502-4f51-8923-c252892ab762 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:38.600363 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11822a72-2502-4f51-8923-c252892ab762 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:38.602636 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:38.621257 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-11822a72-2502-4f51-8923-c252892ab762 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:38.621257 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:38.621257 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:38.621257 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:38.621257 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:38.621257 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:38.621257 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:38.621257 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:38.621257 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:38.621257 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:38.621257 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:38.621257 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:38.621257 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:38.621257 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:38.621257 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:38.622225 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:38.622225 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:38.622225 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:38.622225 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:38.622225 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:38.622225 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:38.622225 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:38.622225 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:38.622225 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:38.622225 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:38.622225 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:38.622225 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:38.622225 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:38.622225 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:38.622225 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:38.622225 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:38.622225 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:38.623283 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:38.623283 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:38.623283 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:38.623283 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:38.623283 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 306/617] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:38.628963 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11822a72-2502-4f51-8923-c252892ab762 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:38.708684 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 312/618] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 06:06:38.710141 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6354449-c834-4a12-9e72-397139ef9ff9 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:38.710437 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6354449-c834-4a12-9e72-397139ef9ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:38.710720 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6354449-c834-4a12-9e72-397139ef9ff9 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:38.717572 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 313/619] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:38.719227 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3b908-a810-4dd4-ba7d-f443df309208 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:38.719516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3b908-a810-4dd4-ba7d-f443df309208 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:38.719798 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3b908-a810-4dd4-ba7d-f443df309208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:38.743269 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-c7c3b908-a810-4dd4-ba7d-f443df309208 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:38.745411 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c7c3b908-a810-4dd4-ba7d-f443df309208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:38.772245 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11822a72-2502-4f51-8923-c252892ab762 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:38.811365 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c753bb-51a5-4c87-9b51-68c45aaa9090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:38.812030 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c753bb-51a5-4c87-9b51-68c45aaa9090 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:38.812314 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c753bb-51a5-4c87-9b51-68c45aaa9090 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:38.812618 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c753bb-51a5-4c87-9b51-68c45aaa9090 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:38.813465 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c753bb-51a5-4c87-9b51-68c45aaa9090 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:38.813772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c753bb-51a5-4c87-9b51-68c45aaa9090 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:38.815149 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c753bb-51a5-4c87-9b51-68c45aaa9090 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:38.823679 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 314/620] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:06:38.824765 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c3b908-a810-4dd4-ba7d-f443df309208 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:38.831946 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 307/621] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:06:38.833867 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c753bb-51a5-4c87-9b51-68c45aaa9090 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:38.891921 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:38.896758 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:38.936299 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c3b908-a810-4dd4-ba7d-f443df309208 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:38.941922 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c753bb-51a5-4c87-9b51-68c45aaa9090 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:38.975734 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf90bd0-3186-4376-b142-2ffc20ce1264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:38.977262 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf90bd0-3186-4376-b142-2ffc20ce1264 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:38.977707 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf90bd0-3186-4376-b142-2ffc20ce1264 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:38.978554 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf90bd0-3186-4376-b142-2ffc20ce1264 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:38.980690 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fb84fb-f15f-4580-ac6c-59ca948a35fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:38.981673 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb84fb-f15f-4580-ac6c-59ca948a35fc None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:38.982102 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb84fb-f15f-4580-ac6c-59ca948a35fc None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:38.982733 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb84fb-f15f-4580-ac6c-59ca948a35fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:38.990454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf90bd0-3186-4376-b142-2ffc20ce1264 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:38.992016 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fb84fb-f15f-4580-ac6c-59ca948a35fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:38.994339 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fb84fb-f15f-4580-ac6c-59ca948a35fc None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:38.999552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-3bf90bd0-3186-4376-b142-2ffc20ce1264 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:06:38.999552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:38.999552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:38.999552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:38.999552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:38.999552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:38.999552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:38.999552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:38.999552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:38.999552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:38.999552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:38.999552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:38.999552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:38.999552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:38.999552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:38.999552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:39.000943 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:39.000943 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:39.000943 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:39.000943 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:39.000943 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:39.000943 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:39.000943 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:39.000943 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:39.000943 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:39.000943 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:39.000943 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:39.000943 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:39.000943 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:39.000943 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:39.000943 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:39.000943 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:39.002135 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:39.002135 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:39.002135 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:39.002135 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:39.002135 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:39.002135 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:39.002135 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:39.002135 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:39.002135 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:39.002135 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:39.002135 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:06:39.002135 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:39.002135 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 315/622] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 18 06:06:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:39.002993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf90bd0-3186-4376-b142-2ffc20ce1264 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:39.013964 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-10fb84fb-f15f-4580-ac6c-59ca948a35fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:39.013964 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:39.013964 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:39.013964 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:39.013964 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:39.013964 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:39.013964 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:39.013964 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:39.013964 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:39.013964 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:39.013964 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:39.013964 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:39.013964 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 06:06:39.013964 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 06:06:39.013964 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 06:06:39.015040 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 06:06:39.015040 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:39.015040 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:39.015040 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:39.015040 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:39.015040 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:39.015040 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:39.015040 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:39.015040 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:39.015040 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:39.015040 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:39.015040 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:39.015040 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:39.015040 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:39.015040 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:39.015040 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:39.015040 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:39.016212 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 06:06:39.016212 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 06:06:39.016212 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 06:06:39.016212 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 06:06:39.016212 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 06:06:39.016212 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 06:06:39.016212 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:39.016212 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:39.022894 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 308/623] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:06:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:06:39.024082 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fb84fb-f15f-4580-ac6c-59ca948a35fc None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:39.121610 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf90bd0-3186-4376-b142-2ffc20ce1264 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:39.138030 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fb84fb-f15f-4580-ac6c-59ca948a35fc None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:39.149269 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:39.156692 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:39.164785 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d49cc-8189-462c-a928-e761d6af7e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:39.166973 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d49cc-8189-462c-a928-e761d6af7e7d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:39.166973 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d49cc-8189-462c-a928-e761d6af7e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:39.167522 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d49cc-8189-462c-a928-e761d6af7e7d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:39.179325 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b6ed22-7c8c-4a25-a4b9-4fb9b0a6f821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:39.180078 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6ed22-7c8c-4a25-a4b9-4fb9b0a6f821 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:39.180478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6ed22-7c8c-4a25-a4b9-4fb9b0a6f821 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:39.180651 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6ed22-7c8c-4a25-a4b9-4fb9b0a6f821 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:39.181466 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b6ed22-7c8c-4a25-a4b9-4fb9b0a6f821 None admin] RBAC: Adding query filter params (name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:39.181779 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b6ed22-7c8c-4a25-a4b9-4fb9b0a6f821 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:39.182289 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d49cc-8189-462c-a928-e761d6af7e7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:39.183627 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b6ed22-7c8c-4a25-a4b9-4fb9b0a6f821 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:39.191223 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-5c5d49cc-8189-462c-a928-e761d6af7e7d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:06:39.191223 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:39.191223 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:39.191223 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:39.191223 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:39.191223 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:39.191223 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:39.191223 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:39.191223 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:39.191223 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:39.191223 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:39.191223 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:39.191223 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:39.191223 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:39.191223 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:39.191223 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:39.192237 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:39.192237 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:39.192237 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:39.192237 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:39.192237 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:39.192237 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:39.192237 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:39.192237 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:39.192237 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:39.192237 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:39.192237 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:39.192237 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:39.192237 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:39.192237 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:39.192237 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:39.192237 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:39.193155 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:39.193155 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:39.193155 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:39.193155 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:39.193155 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:39.193155 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:39.193155 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:39.193155 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:39.193155 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:39.193155 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:39.193155 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:06:39.193155 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:39.195959 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 316/624] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 18 06:06:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:39.196809 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 309/625] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 18 06:06:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:39.201149 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d49cc-8189-462c-a928-e761d6af7e7d None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:39.205529 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b6ed22-7c8c-4a25-a4b9-4fb9b0a6f821 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:39.317066 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d49cc-8189-462c-a928-e761d6af7e7d None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:39.318287 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b6ed22-7c8c-4a25-a4b9-4fb9b0a6f821 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:39.359111 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72700e42-092b-48b9-9c02-9c3c8d0c05c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:39.360356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72700e42-092b-48b9-9c02-9c3c8d0c05c7 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:39.360949 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72700e42-092b-48b9-9c02-9c3c8d0c05c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:39.361431 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72700e42-092b-48b9-9c02-9c3c8d0c05c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:39.361722 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5581ea6-b02c-4433-86e7-529acae7f452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:39.362711 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5581ea6-b02c-4433-86e7-529acae7f452 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:39.363099 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5581ea6-b02c-4433-86e7-529acae7f452 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:39.363284 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72700e42-092b-48b9-9c02-9c3c8d0c05c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:39.363511 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5581ea6-b02c-4433-86e7-529acae7f452 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:39.363998 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72700e42-092b-48b9-9c02-9c3c8d0c05c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:39.364779 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5581ea6-b02c-4433-86e7-529acae7f452 None admin] RBAC: Adding query filter params (name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:39.365332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5581ea6-b02c-4433-86e7-529acae7f452 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:39.366130 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72700e42-092b-48b9-9c02-9c3c8d0c05c7 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:39.367533 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5581ea6-b02c-4433-86e7-529acae7f452 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:39.381374 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 317/626] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 18 06:06:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:39.382594 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72700e42-092b-48b9-9c02-9c3c8d0c05c7 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:39.388707 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 310/627] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 18 06:06:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:39.389717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5581ea6-b02c-4433-86e7-529acae7f452 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:39.433484 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:39.434337 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:39.502699 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72700e42-092b-48b9-9c02-9c3c8d0c05c7 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:39.511880 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5581ea6-b02c-4433-86e7-529acae7f452 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:39.550981 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8f17f0-4954-4ca5-a262-dfdf40a0a0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:39.552124 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f17f0-4954-4ca5-a262-dfdf40a0a0ce None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:39.552593 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f17f0-4954-4ca5-a262-dfdf40a0a0ce None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:39.553016 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f17f0-4954-4ca5-a262-dfdf40a0a0ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:39.559358 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2317828b-645f-4157-a62c-a34a6f02e7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:39.560466 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317828b-645f-4157-a62c-a34a6f02e7e9 None admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:39.560892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317828b-645f-4157-a62c-a34a6f02e7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:39.561272 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317828b-645f-4157-a62c-a34a6f02e7e9 None admin] PATH_INFO: `/v3/projects/8ed50a8a3e184bde9269d7b0d9573d4a/users/0681f5fbea6d415c9ad2f3afae256a5c/roles/ab10744ff77646eab23a769ab89c74aa` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:39.571389 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8f17f0-4954-4ca5-a262-dfdf40a0a0ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:39.573861 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8f17f0-4954-4ca5-a262-dfdf40a0a0ce None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:39.583078 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2317828b-645f-4157-a62c-a34a6f02e7e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ed50a8a3e184bde9269d7b0d9573d4a, user_id=0681f5fbea6d415c9ad2f3afae256a5c, role_id=ab10744ff77646eab23a769ab89c74aa)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:39.585334 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2317828b-645f-4157-a62c-a34a6f02e7e9 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:39.594437 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-ac8f17f0-4954-4ca5-a262-dfdf40a0a0ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:39.594437 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:39.594437 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:39.594437 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:39.594437 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:39.594437 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:39.594437 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:39.594437 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:39.594437 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:39.594437 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:39.594437 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:39.594437 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:39.594437 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:39.594437 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:39.594437 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:39.595790 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:39.595790 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:39.595790 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:39.595790 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:39.595790 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:39.595790 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:39.595790 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:39.595790 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:39.595790 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:39.595790 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:39.595790 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:39.595790 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:39.595790 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:39.595790 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:39.595790 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:39.595790 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:39.595790 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:39.596963 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:39.596963 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:39.596963 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:39.596963 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:39.596963 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 318/628] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:39.597371 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8f17f0-4954-4ca5-a262-dfdf40a0a0ce None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:39.668734 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 311/629] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 18 06:06:39 2022] PUT /identity/v3/projects/8ed50a8a3e184bde9269d7b0d9573d4a/users/0681f5fbea6d415c9ad2f3afae256a5c/roles/ab10744ff77646eab23a769ab89c74aa => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:06:39.670270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2317828b-645f-4157-a62c-a34a6f02e7e9 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:39.731102 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:39.739273 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8f17f0-4954-4ca5-a262-dfdf40a0a0ce None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:39.814715 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697316a6-5dbf-481a-94fa-23f2708cd6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:39.815534 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697316a6-5dbf-481a-94fa-23f2708cd6ed None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:39.815818 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697316a6-5dbf-481a-94fa-23f2708cd6ed None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:39.816093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697316a6-5dbf-481a-94fa-23f2708cd6ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:39.841986 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697316a6-5dbf-481a-94fa-23f2708cd6ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:39.843584 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697316a6-5dbf-481a-94fa-23f2708cd6ed None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:39.877521 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-697316a6-5dbf-481a-94fa-23f2708cd6ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:39.877521 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:39.877521 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:39.877521 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:39.877521 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:39.877521 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:39.877521 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:39.877521 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:39.877521 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:39.877521 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:39.877521 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:39.877521 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:39.877521 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:39.877521 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:39.877521 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:39.878479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:39.878479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:39.878479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:39.878479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:39.878479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:39.878479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:39.878479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:39.878479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:39.878479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:39.878479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:39.878479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:39.878479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:39.878479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:39.878479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:39.878479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:39.878479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:39.878479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:39.879381 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:39.879381 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:39.879381 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:39.879381 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:39.879602 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:39.879674 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 319/630] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:39.887444 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697316a6-5dbf-481a-94fa-23f2708cd6ed None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:39.901517 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2317828b-645f-4157-a62c-a34a6f02e7e9 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:39.944295 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29867e6-38c8-4736-a473-7a06061db6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:39.945853 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29867e6-38c8-4736-a473-7a06061db6a6 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:39.946334 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29867e6-38c8-4736-a473-7a06061db6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:39.946936 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29867e6-38c8-4736-a473-7a06061db6a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:39.948288 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29867e6-38c8-4736-a473-7a06061db6a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:39.948950 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29867e6-38c8-4736-a473-7a06061db6a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:39.950513 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29867e6-38c8-4736-a473-7a06061db6a6 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:39.966414 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 312/631] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:39.972598 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29867e6-38c8-4736-a473-7a06061db6a6 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:40.026812 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697316a6-5dbf-481a-94fa-23f2708cd6ed None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:40.065625 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcecb4f-0a2e-4b60-83bd-f02b75002182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:40.066471 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcecb4f-0a2e-4b60-83bd-f02b75002182 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:40.066763 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcecb4f-0a2e-4b60-83bd-f02b75002182 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:40.067096 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcecb4f-0a2e-4b60-83bd-f02b75002182 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:40.067911 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcecb4f-0a2e-4b60-83bd-f02b75002182 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:40.068230 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcecb4f-0a2e-4b60-83bd-f02b75002182 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:40.069709 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcecb4f-0a2e-4b60-83bd-f02b75002182 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:40.084932 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 320/632] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:06:40.091459 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcecb4f-0a2e-4b60-83bd-f02b75002182 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:40.107073 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:40.189779 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29867e6-38c8-4736-a473-7a06061db6a6 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:40.227375 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a69f06c-e152-4da6-a576-631b5bf5cf1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:40.228877 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69f06c-e152-4da6-a576-631b5bf5cf1b None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:40.229237 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69f06c-e152-4da6-a576-631b5bf5cf1b None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:40.229632 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69f06c-e152-4da6-a576-631b5bf5cf1b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:40.238305 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a69f06c-e152-4da6-a576-631b5bf5cf1b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:40.239754 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a69f06c-e152-4da6-a576-631b5bf5cf1b None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:40.245275 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcecb4f-0a2e-4b60-83bd-f02b75002182 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:40.254941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-8a69f06c-e152-4da6-a576-631b5bf5cf1b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 06:06:40.254941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:40.254941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:40.254941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:40.254941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:40.254941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:40.254941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:40.254941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:40.254941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:40.254941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:40.254941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:40.254941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:40.254941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 06:06:40.254941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 06:06:40.254941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 06:06:40.254941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 06:06:40.256017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:40.256017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:40.256017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 06:06:40.256017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:40.256017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 06:06:40.256017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:40.256017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:40.256017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:40.256017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:40.256017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:40.256017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:40.256017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:40.256017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:40.256017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:40.256017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:40.256017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:40.256017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:40.256017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:40.257949 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:40.257949 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:40.257949 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 06:06:40.257949 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 06:06:40.257949 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 06:06:40.257949 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 06:06:40.257949 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 06:06:40.257949 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 06:06:40.257949 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 06:06:40.257949 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:40.257949 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 313/633] 10.222.0.40 () {56 vars in 1093 bytes} [Mon Jul 18 06:06:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:40.257949 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf921a7e-c5c6-40b6-a030-5109968f624b None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:40.258695 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf921a7e-c5c6-40b6-a030-5109968f624b None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:40.258695 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf921a7e-c5c6-40b6-a030-5109968f624b None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:40.260670 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 314/634] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:40.261827 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d57443-6e32-4535-8b35-2f16efb11aa7 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:40.262196 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d57443-6e32-4535-8b35-2f16efb11aa7 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:40.262483 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d57443-6e32-4535-8b35-2f16efb11aa7 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:40.267778 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 315/635] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:40.267778 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d57443-6e32-4535-8b35-2f16efb11aa7 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:40.278178 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:40.297038 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb232d-00ae-4005-b57d-2cee8ddee453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:40.297846 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb232d-00ae-4005-b57d-2cee8ddee453 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:40.298126 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb232d-00ae-4005-b57d-2cee8ddee453 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:40.298399 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb232d-00ae-4005-b57d-2cee8ddee453 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:40.306762 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb232d-00ae-4005-b57d-2cee8ddee453 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:40.308270 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb232d-00ae-4005-b57d-2cee8ddee453 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:40.323934 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-62bb232d-00ae-4005-b57d-2cee8ddee453 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 06:06:40.323934 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:40.323934 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:40.323934 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:40.323934 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:40.323934 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:40.323934 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:40.323934 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:40.323934 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:40.323934 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:40.323934 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:40.323934 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:40.323934 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 06:06:40.323934 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 06:06:40.323934 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 06:06:40.323934 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 06:06:40.325016 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:40.325016 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:40.325016 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 06:06:40.325016 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:40.325016 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 06:06:40.325016 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:40.325016 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:40.325016 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:40.325016 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:40.325016 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:40.325016 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:40.325016 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:40.325016 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:40.325016 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:40.325016 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:40.325016 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:40.325016 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:40.325016 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:40.325993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:40.325993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:40.325993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 06:06:40.325993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 06:06:40.325993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 06:06:40.325993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 06:06:40.325993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 06:06:40.325993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 06:06:40.325993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 06:06:40.325993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:40.329971 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 321/636] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:06:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:06:40.331311 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f5f38-0e76-4334-9b0c-6a8d3246d33c None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:40.331602 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f5f38-0e76-4334-9b0c-6a8d3246d33c None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:40.332200 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f5f38-0e76-4334-9b0c-6a8d3246d33c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:40.349256 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-207f5f38-0e76-4334-9b0c-6a8d3246d33c None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:40.350447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-207f5f38-0e76-4334-9b0c-6a8d3246d33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:40.383692 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d57443-6e32-4535-8b35-2f16efb11aa7 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:40.419607 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:40.424529 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f0044-c7dd-4d76-a482-aa08d693a9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:40.425182 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f0044-c7dd-4d76-a482-aa08d693a9c2 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:40.436759 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f0044-c7dd-4d76-a482-aa08d693a9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:40.436759 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f0044-c7dd-4d76-a482-aa08d693a9c2 None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:40.436759 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f0044-c7dd-4d76-a482-aa08d693a9c2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:40.436759 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f0044-c7dd-4d76-a482-aa08d693a9c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:40.437140 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 322/637] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:06:40.437227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f0044-c7dd-4d76-a482-aa08d693a9c2 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:40.437319 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee323736-96d6-40ec-a33b-6cbc2cdd69a7 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:40.437319 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee323736-96d6-40ec-a33b-6cbc2cdd69a7 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:40.437319 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee323736-96d6-40ec-a33b-6cbc2cdd69a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:40.448020 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-ee323736-96d6-40ec-a33b-6cbc2cdd69a7 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:40.449260 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ee323736-96d6-40ec-a33b-6cbc2cdd69a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:40.458424 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 316/638] 10.222.0.40 () {56 vars in 1133 bytes} [Mon Jul 18 06:06:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:40.460060 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f0044-c7dd-4d76-a482-aa08d693a9c2 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:40.527841 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 323/639] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:06:40.528665 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee323736-96d6-40ec-a33b-6cbc2cdd69a7 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:40.550081 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:40.591577 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f0044-c7dd-4d76-a482-aa08d693a9c2 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:40.628560 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5786fa-567a-4994-af73-7e0286104050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:40.629237 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5786fa-567a-4994-af73-7e0286104050 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:40.629523 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5786fa-567a-4994-af73-7e0286104050 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:40.629801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5786fa-567a-4994-af73-7e0286104050 None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:40.630645 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5786fa-567a-4994-af73-7e0286104050 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:40.630995 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5786fa-567a-4994-af73-7e0286104050 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:40.632465 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5786fa-567a-4994-af73-7e0286104050 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:40.652584 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 317/640] 10.222.0.40 () {56 vars in 1137 bytes} [Mon Jul 18 06:06:40 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:40.654120 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5786fa-567a-4994-af73-7e0286104050 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:40.659260 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee323736-96d6-40ec-a33b-6cbc2cdd69a7 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:40.691553 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:40.702898 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f36e0e-ab23-42c2-a58f-673786c034f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:40.703895 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f36e0e-ab23-42c2-a58f-673786c034f4 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:40.704251 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f36e0e-ab23-42c2-a58f-673786c034f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:40.704686 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f36e0e-ab23-42c2-a58f-673786c034f4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:40.718097 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f36e0e-ab23-42c2-a58f-673786c034f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:40.730712 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-94f36e0e-ab23-42c2-a58f-673786c034f4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:06:40.730712 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:40.730712 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:40.730712 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:40.730712 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:40.730712 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:40.730712 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:40.730712 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:40.730712 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:40.730712 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:40.730712 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:40.730712 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:40.730712 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:40.730712 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:40.730712 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:40.730712 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:40.731765 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:40.731765 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:40.731765 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:40.731765 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:40.731765 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:40.731765 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:40.731765 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:40.731765 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:40.731765 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:40.731765 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:40.731765 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:40.731765 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:40.731765 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:40.731765 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:40.731765 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:40.731765 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:40.732654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:40.732654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:40.732654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:40.732654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:40.732654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:40.732654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:40.732654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:40.732654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:40.732654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:40.732654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:40.732654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:06:40.732654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:40.732654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 324/641] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 18 06:06:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:40.733367 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f36e0e-ab23-42c2-a58f-673786c034f4 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:40.772739 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5786fa-567a-4994-af73-7e0286104050 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:40.803358 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:40.814362 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cadf110-0b19-49eb-aa10-c98bc6579e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:40.815017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cadf110-0b19-49eb-aa10-c98bc6579e3e None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:40.815301 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cadf110-0b19-49eb-aa10-c98bc6579e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:40.815593 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cadf110-0b19-49eb-aa10-c98bc6579e3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:40.851979 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f36e0e-ab23-42c2-a58f-673786c034f4 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:40.888554 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464ae43-add6-4c5e-9e05-33229c71655e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:40.889367 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464ae43-add6-4c5e-9e05-33229c71655e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:40.889669 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464ae43-add6-4c5e-9e05-33229c71655e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:40.890020 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464ae43-add6-4c5e-9e05-33229c71655e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:40.894365 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cadf110-0b19-49eb-aa10-c98bc6579e3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:40.896653 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cadf110-0b19-49eb-aa10-c98bc6579e3e None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:40.902972 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9464ae43-add6-4c5e-9e05-33229c71655e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:40.912312 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-9464ae43-add6-4c5e-9e05-33229c71655e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:06:40.912312 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:40.912312 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:40.912312 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:40.912312 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:40.912312 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:40.912312 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:40.912312 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:40.912312 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:40.912312 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:40.912312 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:40.912312 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:40.912312 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:40.912312 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:40.912312 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:40.912312 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:40.913646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:40.913646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:40.913646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:40.913646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:40.913646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:40.913646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:40.913646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:40.913646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:40.913646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:40.913646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:40.913646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:40.913646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:40.913646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:40.913646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:40.913646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:40.913646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:40.914510 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:40.914510 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:40.914510 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:40.914510 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:40.914510 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:40.914510 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:40.914510 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:40.914510 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:40.914510 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:40.914510 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:40.914510 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:06:40.914510 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:40.914510 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 325/642] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 18 06:06:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:40.914510 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464ae43-add6-4c5e-9e05-33229c71655e None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:40.977210 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:41.020759 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 318/643] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:06:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 06:06:41.021646 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cadf110-0b19-49eb-aa10-c98bc6579e3e None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:41.042982 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464ae43-add6-4c5e-9e05-33229c71655e None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:41.054502 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:41.082772 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed6fd0-8bf2-4ccc-9dde-7e29498ab811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:41.083437 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed6fd0-8bf2-4ccc-9dde-7e29498ab811 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:41.083729 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed6fd0-8bf2-4ccc-9dde-7e29498ab811 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:41.084012 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed6fd0-8bf2-4ccc-9dde-7e29498ab811 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:41.139128 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cadf110-0b19-49eb-aa10-c98bc6579e3e None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:41.160194 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ed6fd0-8bf2-4ccc-9dde-7e29498ab811 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:41.161964 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ed6fd0-8bf2-4ccc-9dde-7e29498ab811 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:41.175182 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a22bd-519a-4aa9-a240-0b4098f67de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:41.175854 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a22bd-519a-4aa9-a240-0b4098f67de6 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:41.176137 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a22bd-519a-4aa9-a240-0b4098f67de6 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:41.176431 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a22bd-519a-4aa9-a240-0b4098f67de6 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:41.177178 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97a22bd-519a-4aa9-a240-0b4098f67de6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:41.177486 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97a22bd-519a-4aa9-a240-0b4098f67de6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:41.178953 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97a22bd-519a-4aa9-a240-0b4098f67de6 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:41.187792 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 319/644] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 18 06:06:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:41.188641 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a22bd-519a-4aa9-a240-0b4098f67de6 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:41.222203 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:41.280156 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 326/645] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:06:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 06:06:41.281288 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed6fd0-8bf2-4ccc-9dde-7e29498ab811 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:41.295147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:41.313488 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a22bd-519a-4aa9-a240-0b4098f67de6 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:41.357472 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af2c52-51fd-45ac-a989-26d698b823b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:41.358483 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af2c52-51fd-45ac-a989-26d698b823b2 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:41.358941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af2c52-51fd-45ac-a989-26d698b823b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:41.359402 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af2c52-51fd-45ac-a989-26d698b823b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:41.360317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af2c52-51fd-45ac-a989-26d698b823b2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:41.360824 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af2c52-51fd-45ac-a989-26d698b823b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:41.362301 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af2c52-51fd-45ac-a989-26d698b823b2 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:41.376361 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 320/646] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 18 06:06:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:41.377409 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af2c52-51fd-45ac-a989-26d698b823b2 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:41.395278 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed6fd0-8bf2-4ccc-9dde-7e29498ab811 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:41.432579 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1357c8-f930-4a0c-a31c-0cb3d6d5fe13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:41.433387 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1357c8-f930-4a0c-a31c-0cb3d6d5fe13 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:41.433670 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1357c8-f930-4a0c-a31c-0cb3d6d5fe13 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:41.433945 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1357c8-f930-4a0c-a31c-0cb3d6d5fe13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:41.444282 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1357c8-f930-4a0c-a31c-0cb3d6d5fe13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:41.445775 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1357c8-f930-4a0c-a31c-0cb3d6d5fe13 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:41.454241 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:41.465726 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-1a1357c8-f930-4a0c-a31c-0cb3d6d5fe13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:41.465726 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:41.465726 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:41.465726 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:41.465726 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:41.465726 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:41.465726 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:41.465726 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:41.465726 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:41.465726 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:41.465726 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:41.465726 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:41.465726 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:41.465726 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:41.465726 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:41.472975 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:41.472975 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:41.472975 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:41.472975 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:41.472975 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:41.472975 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:41.472975 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:41.472975 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:41.472975 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:41.472975 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:41.472975 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:41.472975 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:41.472975 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:41.472975 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:41.472975 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:41.472975 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:41.472975 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:41.474091 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:41.474091 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:41.474091 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:41.474091 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:41.474091 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 327/647] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:41.474091 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1357c8-f930-4a0c-a31c-0cb3d6d5fe13 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:41.498094 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af2c52-51fd-45ac-a989-26d698b823b2 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:41.536074 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0715a84e-7352-43fe-bb8e-057b04aca5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:41.537281 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715a84e-7352-43fe-bb8e-057b04aca5fb None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:41.537458 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715a84e-7352-43fe-bb8e-057b04aca5fb None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:41.537716 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715a84e-7352-43fe-bb8e-057b04aca5fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:41.549672 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0715a84e-7352-43fe-bb8e-057b04aca5fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:41.551139 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0715a84e-7352-43fe-bb8e-057b04aca5fb None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:41.560457 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:41.580247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-0715a84e-7352-43fe-bb8e-057b04aca5fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:41.580247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:41.580247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:41.580247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:41.580247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:41.580247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:41.580247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:41.580247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:41.580247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:41.580247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:41.580247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:41.580247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:41.580247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:41.580247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:41.580247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:41.581188 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:41.581188 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:41.581188 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:41.581188 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:41.581188 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:41.581188 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:41.581188 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:41.581188 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:41.581188 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:41.581188 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:41.581188 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:41.581188 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:41.581188 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:41.581188 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:41.581188 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:41.581188 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:41.581188 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:41.582101 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:41.582101 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:41.582101 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:41.582101 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:41.582101 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 321/648] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:41.582101 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0715a84e-7352-43fe-bb8e-057b04aca5fb None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:41.590389 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1357c8-f930-4a0c-a31c-0cb3d6d5fe13 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:41.627467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2715fe83-f820-456d-9ea0-bcc3258d7d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:41.628299 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715fe83-f820-456d-9ea0-bcc3258d7d67 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:41.628605 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715fe83-f820-456d-9ea0-bcc3258d7d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:41.628885 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715fe83-f820-456d-9ea0-bcc3258d7d67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:41.640530 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2715fe83-f820-456d-9ea0-bcc3258d7d67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:41.642121 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2715fe83-f820-456d-9ea0-bcc3258d7d67 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:41.664320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-2715fe83-f820-456d-9ea0-bcc3258d7d67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:41.664320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:41.664320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:41.664320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:41.664320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:41.664320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:41.664320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:41.664320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:41.664320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:41.664320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:41.664320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:41.664320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:41.664320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:41.664320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:41.664320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:41.665377 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:41.665377 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:41.665377 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:41.665377 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:41.665377 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:41.665377 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:41.665377 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:41.665377 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:41.665377 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:41.665377 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:41.665377 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:41.665377 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:41.665377 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:41.665377 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:41.665377 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:41.665377 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:41.665377 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:41.666263 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:41.666263 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:41.666263 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:41.666263 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:41.668775 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 328/649] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:41.670193 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2715fe83-f820-456d-9ea0-bcc3258d7d67 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:41.700164 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0715a84e-7352-43fe-bb8e-057b04aca5fb None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:41.738706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8f6b0-34ec-4841-8769-6ebb53c9a15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:41.739542 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8f6b0-34ec-4841-8769-6ebb53c9a15f None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:41.739828 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8f6b0-34ec-4841-8769-6ebb53c9a15f None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:41.739923 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:41.740108 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8f6b0-34ec-4841-8769-6ebb53c9a15f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:41.753488 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff8f6b0-34ec-4841-8769-6ebb53c9a15f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:41.757060 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff8f6b0-34ec-4841-8769-6ebb53c9a15f None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:41.775944 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-eff8f6b0-34ec-4841-8769-6ebb53c9a15f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:41.775944 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:41.775944 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:41.775944 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:41.775944 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:41.775944 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:41.775944 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:41.775944 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:41.775944 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:41.775944 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:41.775944 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:41.775944 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:41.775944 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:41.775944 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:41.775944 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:41.777066 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:41.777066 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:41.777066 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:41.777066 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:41.777066 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:41.777066 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:41.777066 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:41.777066 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:41.777066 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:41.777066 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:41.777066 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:41.777066 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:41.777066 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:41.777066 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:41.777066 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:41.777066 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:41.777066 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:41.778161 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:41.778161 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:41.778161 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:41.778161 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:41.779799 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 322/650] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:41.780832 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8f6b0-34ec-4841-8769-6ebb53c9a15f None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:41.808914 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2715fe83-f820-456d-9ea0-bcc3258d7d67 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:41.844314 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:41.849276 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180c559c-f03c-4e9b-8311-dab390eca3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:41.849958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c559c-f03c-4e9b-8311-dab390eca3a3 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:41.850251 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c559c-f03c-4e9b-8311-dab390eca3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:41.850523 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c559c-f03c-4e9b-8311-dab390eca3a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:41.851262 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180c559c-f03c-4e9b-8311-dab390eca3a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:41.851569 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180c559c-f03c-4e9b-8311-dab390eca3a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:41.853625 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180c559c-f03c-4e9b-8311-dab390eca3a3 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:41.871909 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 329/651] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:41.871909 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180c559c-f03c-4e9b-8311-dab390eca3a3 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:41.903320 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8f6b0-34ec-4841-8769-6ebb53c9a15f None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:41.945618 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e082e-67c1-4aff-8881-5a087fdbbf50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:41.946306 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e082e-67c1-4aff-8881-5a087fdbbf50 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:41.947251 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e082e-67c1-4aff-8881-5a087fdbbf50 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:41.947566 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e082e-67c1-4aff-8881-5a087fdbbf50 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:41.948317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3e082e-67c1-4aff-8881-5a087fdbbf50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:41.948659 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3e082e-67c1-4aff-8881-5a087fdbbf50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:41.950056 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3e082e-67c1-4aff-8881-5a087fdbbf50 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:41.963611 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 323/652] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:41.964527 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e082e-67c1-4aff-8881-5a087fdbbf50 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:41.982870 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180c559c-f03c-4e9b-8311-dab390eca3a3 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:41.994552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:42.018451 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222185c1-865f-478f-a6e0-0874b2ab53c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:42.019172 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222185c1-865f-478f-a6e0-0874b2ab53c6 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:42.019457 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222185c1-865f-478f-a6e0-0874b2ab53c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:42.019736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222185c1-865f-478f-a6e0-0874b2ab53c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:42.020493 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222185c1-865f-478f-a6e0-0874b2ab53c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:42.020804 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222185c1-865f-478f-a6e0-0874b2ab53c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:42.022193 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222185c1-865f-478f-a6e0-0874b2ab53c6 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:42.035568 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 330/653] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:42.036404 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222185c1-865f-478f-a6e0-0874b2ab53c6 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:42.071291 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e082e-67c1-4aff-8881-5a087fdbbf50 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:42.085593 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:42.106962 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7fd0f1-9233-48ea-9f58-d52738889aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:42.107568 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fd0f1-9233-48ea-9f58-d52738889aaa None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:42.107849 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fd0f1-9233-48ea-9f58-d52738889aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:42.108121 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fd0f1-9233-48ea-9f58-d52738889aaa None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:42.108888 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7fd0f1-9233-48ea-9f58-d52738889aaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:42.109189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7fd0f1-9233-48ea-9f58-d52738889aaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:42.110601 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7fd0f1-9233-48ea-9f58-d52738889aaa None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:42.123011 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 324/654] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:42.124328 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d134592-277e-4294-a845-bb6121b3aef9 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:42.124642 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d134592-277e-4294-a845-bb6121b3aef9 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:42.124921 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d134592-277e-4294-a845-bb6121b3aef9 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:42.128452 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 325/655] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:42.128452 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d134592-277e-4294-a845-bb6121b3aef9 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:42.143248 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222185c1-865f-478f-a6e0-0874b2ab53c6 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:42.177108 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4b1c1-d656-4b2c-bef1-2786fcb33e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:42.177913 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4b1c1-d656-4b2c-bef1-2786fcb33e1b None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:42.178204 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4b1c1-d656-4b2c-bef1-2786fcb33e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:42.178486 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4b1c1-d656-4b2c-bef1-2786fcb33e1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:42.186297 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc4b1c1-d656-4b2c-bef1-2786fcb33e1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:42.187760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc4b1c1-d656-4b2c-bef1-2786fcb33e1b None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:42.203014 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-4dc4b1c1-d656-4b2c-bef1-2786fcb33e1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:42.203014 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:42.203014 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:42.203014 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:42.203014 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:42.203014 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:42.203014 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:42.203014 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:42.203014 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:42.203014 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:42.203014 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:42.203014 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:42.203014 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 06:06:42.203014 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 06:06:42.203014 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 06:06:42.204027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 06:06:42.204027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:42.204027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:42.204027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:42.204027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:42.204027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:42.204027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:42.204027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:42.204027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:42.204027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:42.204027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:42.204027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:42.204027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:42.204027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:42.204027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:42.204027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:42.204027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:42.204933 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 06:06:42.204933 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 06:06:42.204933 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 06:06:42.204933 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 06:06:42.204933 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 06:06:42.204933 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 06:06:42.204933 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:42.204933 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:42.204933 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 331/656] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:06:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:42.205441 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4b1c1-d656-4b2c-bef1-2786fcb33e1b None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:42.232349 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d134592-277e-4294-a845-bb6121b3aef9 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:42.236920 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:42.267213 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e889dcab-b3b0-4fa2-a576-44790c49adc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:42.268048 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889dcab-b3b0-4fa2-a576-44790c49adc2 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:42.268332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889dcab-b3b0-4fa2-a576-44790c49adc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:42.268647 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889dcab-b3b0-4fa2-a576-44790c49adc2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:42.276553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e889dcab-b3b0-4fa2-a576-44790c49adc2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:42.278028 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e889dcab-b3b0-4fa2-a576-44790c49adc2 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:42.293212 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-e889dcab-b3b0-4fa2-a576-44790c49adc2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 06:06:42.293212 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:42.293212 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:42.293212 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:42.293212 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:42.293212 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:42.293212 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:42.293212 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:42.293212 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:42.293212 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:42.293212 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:42.293212 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:42.293212 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 06:06:42.293212 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 06:06:42.293212 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 06:06:42.293212 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 06:06:42.294439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:42.294439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:42.294439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 06:06:42.294439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:42.294439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 06:06:42.294439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:42.294439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:42.294439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:42.294439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:42.294439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:42.294439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:42.294439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:42.294439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:42.294439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:42.294439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:42.294439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:42.294439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:42.294439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:42.295681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:42.295681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:42.295681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 06:06:42.295681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 06:06:42.295681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 06:06:42.295681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 06:06:42.295681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 06:06:42.295681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 06:06:42.295681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 06:06:42.295681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:42.295681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 326/657] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 18 06:06:42 2022] GET /identity/v3/users/neutron => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:42.295681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e889dcab-b3b0-4fa2-a576-44790c49adc2 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:42.316921 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4b1c1-d656-4b2c-bef1-2786fcb33e1b None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:42.329989 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:42.354149 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6844ca6-a894-4a7b-bae8-3719b7f6efeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:42.354955 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6844ca6-a894-4a7b-bae8-3719b7f6efeb None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:42.355246 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6844ca6-a894-4a7b-bae8-3719b7f6efeb None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:42.355524 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6844ca6-a894-4a7b-bae8-3719b7f6efeb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:42.362771 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6844ca6-a894-4a7b-bae8-3719b7f6efeb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:42.364175 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6844ca6-a894-4a7b-bae8-3719b7f6efeb None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:42.379066 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-e6844ca6-a894-4a7b-bae8-3719b7f6efeb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:42.379066 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:42.379066 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:42.379066 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:42.379066 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:42.379066 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:42.379066 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:42.379066 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:42.379066 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:42.379066 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:42.379066 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:42.379066 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:42.379066 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 06:06:42.379066 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 06:06:42.379066 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 06:06:42.380185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 06:06:42.380185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:42.380185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:42.380185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:42.380185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:42.380185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:42.380185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:42.380185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:42.380185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:42.380185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:42.380185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:42.380185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:42.380185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:42.380185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:42.380185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:42.380185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:42.380185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:42.381145 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 06:06:42.381145 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 06:06:42.381145 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 06:06:42.381145 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 06:06:42.381145 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 06:06:42.381145 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 06:06:42.381145 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:42.381145 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:42.384562 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 332/658] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:06:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:42.386106 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90104e1-0417-4924-b381-19819387b909 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:42.386403 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90104e1-0417-4924-b381-19819387b909 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:42.386685 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90104e1-0417-4924-b381-19819387b909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:42.403908 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-d90104e1-0417-4924-b381-19819387b909 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:42.404104 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e889dcab-b3b0-4fa2-a576-44790c49adc2 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:42.405231 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d90104e1-0417-4924-b381-19819387b909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:42.439172 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe3749-e906-46d3-b09c-b2e0cdb97621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:42.439974 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe3749-e906-46d3-b09c-b2e0cdb97621 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:42.440263 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe3749-e906-46d3-b09c-b2e0cdb97621 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:42.440572 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe3749-e906-46d3-b09c-b2e0cdb97621 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:42.448437 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efe3749-e906-46d3-b09c-b2e0cdb97621 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:42.449909 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efe3749-e906-46d3-b09c-b2e0cdb97621 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:42.464876 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-7efe3749-e906-46d3-b09c-b2e0cdb97621 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 18 06:06:42.464876 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:42.464876 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:42.464876 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:42.464876 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:42.464876 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:42.464876 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:42.464876 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:42.464876 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:42.464876 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:42.464876 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:42.464876 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:42.464876 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 06:06:42.464876 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 06:06:42.464876 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 06:06:42.464876 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 06:06:42.465925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:42.465925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:42.465925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 06:06:42.465925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:42.465925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 06:06:42.465925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:42.465925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:42.465925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:42.465925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:42.465925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:42.465925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:42.465925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:42.465925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:42.465925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:42.465925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:42.465925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:42.465925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:42.465925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:42.466912 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:42.466912 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:42.466912 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 06:06:42.466912 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 06:06:42.466912 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 06:06:42.466912 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 06:06:42.466912 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 06:06:42.466912 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 06:06:42.466912 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 18 06:06:42.466912 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:42.466912 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 327/659] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:06:42 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:42.466912 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe3749-e906-46d3-b09c-b2e0cdb97621 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:42.476826 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 333/660] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:06:42.478496 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90104e1-0417-4924-b381-19819387b909 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:42.510026 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:42.562311 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:42.570709 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe3749-e906-46d3-b09c-b2e0cdb97621 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:42.594314 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90104e1-0417-4924-b381-19819387b909 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:42.609512 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb318a-5faf-483a-a57a-c7397cac364d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:42.610175 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb318a-5faf-483a-a57a-c7397cac364d None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:42.610458 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb318a-5faf-483a-a57a-c7397cac364d None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:42.610735 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb318a-5faf-483a-a57a-c7397cac364d None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:42.611526 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beb318a-5faf-483a-a57a-c7397cac364d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:42.611833 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beb318a-5faf-483a-a57a-c7397cac364d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:42.613344 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beb318a-5faf-483a-a57a-c7397cac364d None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:42.626173 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 328/661] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 18 06:06:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:42.627055 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb318a-5faf-483a-a57a-c7397cac364d None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:42.636856 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba38188-0947-4867-8bcc-1ce86d28d509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:42.637563 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba38188-0947-4867-8bcc-1ce86d28d509 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:42.637853 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba38188-0947-4867-8bcc-1ce86d28d509 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:42.638136 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba38188-0947-4867-8bcc-1ce86d28d509 None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:42.639002 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba38188-0947-4867-8bcc-1ce86d28d509 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:42.639387 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba38188-0947-4867-8bcc-1ce86d28d509 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:42.640963 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba38188-0947-4867-8bcc-1ce86d28d509 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:42.663022 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 334/662] 10.222.0.40 () {56 vars in 1139 bytes} [Mon Jul 18 06:06:42 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:42.664114 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba38188-0947-4867-8bcc-1ce86d28d509 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:42.739230 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb318a-5faf-483a-a57a-c7397cac364d None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:42.746724 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:42.768889 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba38188-0947-4867-8bcc-1ce86d28d509 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:42.774147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa867c-b49b-40e8-9827-154420fb8259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:42.775437 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa867c-b49b-40e8-9827-154420fb8259 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:42.775724 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa867c-b49b-40e8-9827-154420fb8259 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:42.776012 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa867c-b49b-40e8-9827-154420fb8259 None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:42.776985 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaa867c-b49b-40e8-9827-154420fb8259 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:42.777297 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaa867c-b49b-40e8-9827-154420fb8259 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:42.778731 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaa867c-b49b-40e8-9827-154420fb8259 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:42.792262 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 329/663] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 18 06:06:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:42.793161 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa867c-b49b-40e8-9827-154420fb8259 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:42.806935 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de5232-6388-4a37-98f3-262771756533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:42.807779 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de5232-6388-4a37-98f3-262771756533 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:42.808122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de5232-6388-4a37-98f3-262771756533 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:42.808406 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de5232-6388-4a37-98f3-262771756533 None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:42.809264 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:42.809409 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1de5232-6388-4a37-98f3-262771756533 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:42.809967 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1de5232-6388-4a37-98f3-262771756533 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:42.811072 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1de5232-6388-4a37-98f3-262771756533 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:42.829484 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 335/664] 10.222.0.40 () {56 vars in 1137 bytes} [Mon Jul 18 06:06:42 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:42.830329 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de5232-6388-4a37-98f3-262771756533 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:42.896392 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa867c-b49b-40e8-9827-154420fb8259 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:42.931909 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a372815c-251a-4834-8a0f-c73b03e6676d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:42.932980 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372815c-251a-4834-8a0f-c73b03e6676d None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:42.933268 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372815c-251a-4834-8a0f-c73b03e6676d None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:42.933547 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372815c-251a-4834-8a0f-c73b03e6676d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:42.937651 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de5232-6388-4a37-98f3-262771756533 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:42.943818 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a372815c-251a-4834-8a0f-c73b03e6676d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:42.951925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-a372815c-251a-4834-8a0f-c73b03e6676d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:06:42.951925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:42.951925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:42.951925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:42.951925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:42.951925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:42.951925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:42.951925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:42.951925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:42.951925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:42.951925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:42.951925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:42.951925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:42.951925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:42.951925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:42.951925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:42.952918 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:42.952918 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:42.952918 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:42.952918 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:42.952918 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:42.952918 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:42.952918 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:42.952918 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:42.952918 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:42.952918 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:42.952918 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:42.952918 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:42.952918 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:42.952918 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:42.952918 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:42.952918 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:42.954032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:42.954032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:42.954032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:42.954032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:42.954032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:42.954032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:42.954032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:42.954032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:42.954032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:42.954032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:42.954032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:06:42.954032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:42.954032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 330/665] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 18 06:06:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:42.954032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a372815c-251a-4834-8a0f-c73b03e6676d None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:42.978626 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74148af0-bdfa-46cf-a55c-ad6b738e03c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:42.979298 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74148af0-bdfa-46cf-a55c-ad6b738e03c8 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:42.979584 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74148af0-bdfa-46cf-a55c-ad6b738e03c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:42.979861 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74148af0-bdfa-46cf-a55c-ad6b738e03c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:42.980742 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74148af0-bdfa-46cf-a55c-ad6b738e03c8 None admin] RBAC: Adding query filter params (role.id=ab10744ff77646eab23a769ab89c74aa, scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a, user.id=f92d948810ea4241810473506b654f7a) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:42.981059 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74148af0-bdfa-46cf-a55c-ad6b738e03c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:42.982527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74148af0-bdfa-46cf-a55c-ad6b738e03c8 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:42.985704 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:42.989220 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 336/666] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:06:42 2022] GET /identity/v3/role_assignments?user.id=f92d948810ea4241810473506b654f7a&scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a&role.id=ab10744ff77646eab23a769ab89c74aa => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:42.990022 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74148af0-bdfa-46cf-a55c-ad6b738e03c8 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:43.033730 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:43.071895 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a372815c-251a-4834-8a0f-c73b03e6676d None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:43.113477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ff3d7-f9d4-41c9-af53-d5076eddf6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:43.114150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ff3d7-f9d4-41c9-af53-d5076eddf6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:43.114427 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ff3d7-f9d4-41c9-af53-d5076eddf6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:43.114737 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ff3d7-f9d4-41c9-af53-d5076eddf6f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:43.120727 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74148af0-bdfa-46cf-a55c-ad6b738e03c8 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:43.167744 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f9e80-11f3-42f2-9ff5-417bde6c1785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:43.168535 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f9e80-11f3-42f2-9ff5-417bde6c1785 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:43.168671 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f9e80-11f3-42f2-9ff5-417bde6c1785 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:43.168954 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f9e80-11f3-42f2-9ff5-417bde6c1785 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:43.169801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1f9e80-11f3-42f2-9ff5-417bde6c1785 None admin] RBAC: Adding query filter params (role.id=ab10744ff77646eab23a769ab89c74aa, scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a, user.id=5af3a4a1bc1f4c4e9653d0a790f12606) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:43.170112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1f9e80-11f3-42f2-9ff5-417bde6c1785 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:43.171554 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1f9e80-11f3-42f2-9ff5-417bde6c1785 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:43.180516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 337/667] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:06:42 2022] GET /identity/v3/role_assignments?user.id=5af3a4a1bc1f4c4e9653d0a790f12606&scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a&role.id=ab10744ff77646eab23a769ab89c74aa => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:43.181371 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f9e80-11f3-42f2-9ff5-417bde6c1785 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:43.200584 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37ff3d7-f9d4-41c9-af53-d5076eddf6f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:43.202220 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37ff3d7-f9d4-41c9-af53-d5076eddf6f2 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:43.273945 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:43.313148 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:43.328259 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f9e80-11f3-42f2-9ff5-417bde6c1785 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:43.359445 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 331/668] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:06:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 06:06:43.360703 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ff3d7-f9d4-41c9-af53-d5076eddf6f2 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:43.377987 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b92615b-c034-43da-b66b-6d88ff6f436c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:43.378654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92615b-c034-43da-b66b-6d88ff6f436c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:43.378954 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92615b-c034-43da-b66b-6d88ff6f436c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:43.379236 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92615b-c034-43da-b66b-6d88ff6f436c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:43.472216 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b92615b-c034-43da-b66b-6d88ff6f436c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:43.473686 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b92615b-c034-43da-b66b-6d88ff6f436c None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:43.487772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ff3d7-f9d4-41c9-af53-d5076eddf6f2 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:43.527447 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06bdfd7-2540-4fc7-b60d-8586d0df7ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:43.528548 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06bdfd7-2540-4fc7-b60d-8586d0df7ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:43.529040 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06bdfd7-2540-4fc7-b60d-8586d0df7ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:43.529517 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06bdfd7-2540-4fc7-b60d-8586d0df7ea4 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:43.530514 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06bdfd7-2540-4fc7-b60d-8586d0df7ea4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:43.531057 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06bdfd7-2540-4fc7-b60d-8586d0df7ea4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:43.532702 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06bdfd7-2540-4fc7-b60d-8586d0df7ea4 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:43.542895 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 332/669] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 18 06:06:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:43.544098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06bdfd7-2540-4fc7-b60d-8586d0df7ea4 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:43.544899 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:43.584919 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:43.585927 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 338/670] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:06:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 06:06:43.588116 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b92615b-c034-43da-b66b-6d88ff6f436c None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:43.669763 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06bdfd7-2540-4fc7-b60d-8586d0df7ea4 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:43.700443 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b92615b-c034-43da-b66b-6d88ff6f436c None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:43.714334 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df218ad-a048-4741-9d61-5fb3b6a7a6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:43.715637 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df218ad-a048-4741-9d61-5fb3b6a7a6de None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:43.716125 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df218ad-a048-4741-9d61-5fb3b6a7a6de None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:43.716697 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df218ad-a048-4741-9d61-5fb3b6a7a6de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:43.730356 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df218ad-a048-4741-9d61-5fb3b6a7a6de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:43.732232 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df218ad-a048-4741-9d61-5fb3b6a7a6de None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:43.738690 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e546a-24fc-4c2a-b1ec-9bd8ba78f359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:43.739497 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e546a-24fc-4c2a-b1ec-9bd8ba78f359 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:43.739784 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e546a-24fc-4c2a-b1ec-9bd8ba78f359 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:43.740064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e546a-24fc-4c2a-b1ec-9bd8ba78f359 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:43.751604 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8e546a-24fc-4c2a-b1ec-9bd8ba78f359 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:43.753103 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8e546a-24fc-4c2a-b1ec-9bd8ba78f359 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:43.754695 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-4df218ad-a048-4741-9d61-5fb3b6a7a6de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:43.754695 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:43.754695 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:43.754695 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:43.754695 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:43.754695 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:43.754695 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:43.754695 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:43.754695 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:43.754695 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:43.754695 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:43.754695 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:43.754695 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:43.754695 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:43.754695 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:43.755610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:43.755610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:43.755610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:43.755610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:43.755610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:43.755610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:43.755610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:43.755610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:43.755610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:43.755610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:43.755610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:43.755610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:43.755610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:43.755610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:43.755610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:43.755610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:43.755610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:43.756547 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:43.756547 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:43.756547 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:43.756547 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:43.763636 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 333/671] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:43.765128 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df218ad-a048-4741-9d61-5fb3b6a7a6de None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:43.772273 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-4f8e546a-24fc-4c2a-b1ec-9bd8ba78f359 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:43.772273 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:43.772273 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:43.772273 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:43.772273 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:43.772273 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:43.772273 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:43.772273 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:43.772273 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:43.772273 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:43.772273 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:43.772273 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:43.772273 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:43.772273 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:43.772273 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:43.773447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:43.773447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:43.773447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:43.773447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:43.773447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:43.773447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:43.773447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:43.773447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:43.773447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:43.773447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:43.773447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:43.773447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:43.773447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:43.773447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:43.773447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:43.773447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:43.773447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:43.775062 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:43.775062 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:43.775062 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:43.775062 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:43.775062 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 339/672] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:43.775531 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e546a-24fc-4c2a-b1ec-9bd8ba78f359 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:43.800849 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:43.863059 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:43.879475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df218ad-a048-4741-9d61-5fb3b6a7a6de None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:43.888775 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e546a-24fc-4c2a-b1ec-9bd8ba78f359 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:43.918530 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e392810-94d1-4403-aa07-da6621b3aacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:43.919418 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e392810-94d1-4403-aa07-da6621b3aacb None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:43.919769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e392810-94d1-4403-aa07-da6621b3aacb None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:43.920110 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e392810-94d1-4403-aa07-da6621b3aacb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:43.925948 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6136cf3-94a4-4e4d-9cb4-e1cf54d93f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:43.926621 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6136cf3-94a4-4e4d-9cb4-e1cf54d93f11 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:43.926913 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6136cf3-94a4-4e4d-9cb4-e1cf54d93f11 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:43.927199 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6136cf3-94a4-4e4d-9cb4-e1cf54d93f11 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:43.927985 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6136cf3-94a4-4e4d-9cb4-e1cf54d93f11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:43.928338 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6136cf3-94a4-4e4d-9cb4-e1cf54d93f11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:43.930818 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6136cf3-94a4-4e4d-9cb4-e1cf54d93f11 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:43.933860 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e392810-94d1-4403-aa07-da6621b3aacb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:43.935373 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e392810-94d1-4403-aa07-da6621b3aacb None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:43.943778 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 340/673] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:43.944786 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6136cf3-94a4-4e4d-9cb4-e1cf54d93f11 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:43.960065 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-6e392810-94d1-4403-aa07-da6621b3aacb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:43.960065 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:43.960065 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:43.960065 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:43.960065 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:43.960065 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:43.960065 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:43.960065 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:43.960065 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:43.960065 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:43.960065 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:43.960065 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:43.960065 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:43.960065 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:43.960065 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:43.961058 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:43.961058 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:43.961058 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:43.961058 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:43.961058 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:43.961058 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:43.961058 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:43.961058 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:43.961058 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:43.961058 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:43.961058 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:43.961058 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:43.961058 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:43.961058 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:43.961058 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:43.961058 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:43.961058 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:43.962546 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:43.962546 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:43.962546 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:43.962546 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:43.962546 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 334/674] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:43.963463 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e392810-94d1-4403-aa07-da6621b3aacb None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:44.058220 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:44.089398 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6136cf3-94a4-4e4d-9cb4-e1cf54d93f11 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:44.099382 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e392810-94d1-4403-aa07-da6621b3aacb None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:44.136577 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de43178f-3d20-4d57-82ab-b3a9b08b42ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:44.137259 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43178f-3d20-4d57-82ab-b3a9b08b42ff None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:44.137801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43178f-3d20-4d57-82ab-b3a9b08b42ff None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:44.138096 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43178f-3d20-4d57-82ab-b3a9b08b42ff None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:44.139262 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de43178f-3d20-4d57-82ab-b3a9b08b42ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:44.139577 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de43178f-3d20-4d57-82ab-b3a9b08b42ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:44.141504 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de43178f-3d20-4d57-82ab-b3a9b08b42ff None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:44.148167 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5aa046-3455-4654-89fa-4f7b79abe177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:44.149235 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5aa046-3455-4654-89fa-4f7b79abe177 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:44.150313 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5aa046-3455-4654-89fa-4f7b79abe177 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:44.150772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5aa046-3455-4654-89fa-4f7b79abe177 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:44.156803 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 341/675] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:44.157718 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de43178f-3d20-4d57-82ab-b3a9b08b42ff None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:44.159765 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5aa046-3455-4654-89fa-4f7b79abe177 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:44.163237 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5aa046-3455-4654-89fa-4f7b79abe177 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:44.166134 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:44.185331 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-ac5aa046-3455-4654-89fa-4f7b79abe177 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:44.185331 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:44.185331 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:44.185331 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:44.185331 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:44.185331 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:44.185331 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:44.185331 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:44.185331 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:44.185331 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:44.185331 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:44.185331 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:44.185331 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 06:06:44.185331 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 06:06:44.185331 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 06:06:44.186469 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 06:06:44.186469 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:44.186469 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:44.186469 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:44.186469 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:44.186469 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:44.186469 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:44.186469 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:44.186469 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:44.186469 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:44.186469 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:44.186469 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:44.186469 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:44.186469 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:44.186469 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:44.186469 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:44.186469 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:44.187585 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 06:06:44.187585 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 06:06:44.187585 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 06:06:44.187585 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 06:06:44.187585 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 06:06:44.187585 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 06:06:44.187585 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:44.187585 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:44.190307 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 335/676] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:06:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:44.191772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5aa046-3455-4654-89fa-4f7b79abe177 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:44.293966 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de43178f-3d20-4d57-82ab-b3a9b08b42ff None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:44.334602 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64140b4-f8dc-4904-82e5-d0224fb5e759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:44.335227 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64140b4-f8dc-4904-82e5-d0224fb5e759 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:44.335524 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64140b4-f8dc-4904-82e5-d0224fb5e759 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:44.336005 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64140b4-f8dc-4904-82e5-d0224fb5e759 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:44.336559 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64140b4-f8dc-4904-82e5-d0224fb5e759 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:44.336877 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64140b4-f8dc-4904-82e5-d0224fb5e759 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:44.338922 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64140b4-f8dc-4904-82e5-d0224fb5e759 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:44.342206 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:44.349841 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5aa046-3455-4654-89fa-4f7b79abe177 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:44.360144 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 342/677] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:44.363001 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64140b4-f8dc-4904-82e5-d0224fb5e759 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:44.391456 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08238ed-9e81-4ea6-b35d-f3738b223f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:44.392259 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08238ed-9e81-4ea6-b35d-f3738b223f04 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:44.392582 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08238ed-9e81-4ea6-b35d-f3738b223f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:44.392943 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08238ed-9e81-4ea6-b35d-f3738b223f04 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:44.402001 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08238ed-9e81-4ea6-b35d-f3738b223f04 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:44.403604 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08238ed-9e81-4ea6-b35d-f3738b223f04 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:44.420161 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-f08238ed-9e81-4ea6-b35d-f3738b223f04 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 06:06:44.420161 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:44.420161 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:44.420161 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:44.420161 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:44.420161 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:44.420161 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:44.420161 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:44.420161 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:44.420161 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:44.420161 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:44.420161 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:44.420161 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 06:06:44.420161 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 06:06:44.420161 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 06:06:44.420161 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 06:06:44.421220 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:44.421220 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:44.421220 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 06:06:44.421220 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:44.421220 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 06:06:44.421220 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:44.421220 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:44.421220 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:44.421220 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:44.421220 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:44.421220 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:44.421220 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:44.421220 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:44.421220 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:44.421220 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:44.421220 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:44.421220 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:44.421220 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:44.422158 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:44.422158 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:44.422158 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 06:06:44.422158 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 06:06:44.422158 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 06:06:44.422158 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 06:06:44.422158 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 06:06:44.422158 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 06:06:44.422158 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 06:06:44.422158 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:44.422158 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 336/678] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 18 06:06:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:44.422767 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08238ed-9e81-4ea6-b35d-f3738b223f04 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:44.498835 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:44.513312 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64140b4-f8dc-4904-82e5-d0224fb5e759 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:44.544583 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08238ed-9e81-4ea6-b35d-f3738b223f04 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:44.560937 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e66a8-e56b-4e18-a729-a533094a22d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:44.562176 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e66a8-e56b-4e18-a729-a533094a22d9 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:44.563144 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e66a8-e56b-4e18-a729-a533094a22d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:44.563620 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e66a8-e56b-4e18-a729-a533094a22d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:44.564961 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e66a8-e56b-4e18-a729-a533094a22d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:44.565457 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e66a8-e56b-4e18-a729-a533094a22d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:44.567788 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e66a8-e56b-4e18-a729-a533094a22d9 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:44.586355 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de4550-2124-4a1f-bfc0-2ff5aaa90879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:44.587265 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de4550-2124-4a1f-bfc0-2ff5aaa90879 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:44.587660 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de4550-2124-4a1f-bfc0-2ff5aaa90879 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:44.588029 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de4550-2124-4a1f-bfc0-2ff5aaa90879 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:44.588232 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 343/679] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 18 06:06:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:44.589319 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e66a8-e56b-4e18-a729-a533094a22d9 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:44.596229 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3de4550-2124-4a1f-bfc0-2ff5aaa90879 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:44.597849 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3de4550-2124-4a1f-bfc0-2ff5aaa90879 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:44.613553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-d3de4550-2124-4a1f-bfc0-2ff5aaa90879 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:44.613553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:44.613553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:44.613553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:44.613553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:44.613553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:44.613553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:44.613553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:44.613553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:44.613553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:44.613553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:44.613553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:44.613553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 06:06:44.613553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 06:06:44.613553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 06:06:44.614720 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 06:06:44.614720 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:44.614720 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:44.614720 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:44.614720 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:44.614720 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:44.614720 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:44.614720 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:44.614720 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:44.614720 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:44.614720 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:44.614720 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:44.614720 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:44.614720 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:44.614720 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:44.614720 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:44.614720 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:44.615963 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 06:06:44.615963 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 06:06:44.615963 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 06:06:44.615963 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 06:06:44.615963 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 06:06:44.615963 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 06:06:44.615963 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:44.615963 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:44.622627 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 337/680] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:06:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:44.623628 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de4550-2124-4a1f-bfc0-2ff5aaa90879 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:44.665687 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:44.717408 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e66a8-e56b-4e18-a729-a533094a22d9 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:44.751477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de4550-2124-4a1f-bfc0-2ff5aaa90879 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:44.754411 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4385c-b0a9-4033-8b81-75bd2c812d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:44.755419 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4385c-b0a9-4033-8b81-75bd2c812d2d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:44.755623 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4385c-b0a9-4033-8b81-75bd2c812d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:44.756007 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4385c-b0a9-4033-8b81-75bd2c812d2d None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:44.756995 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f4385c-b0a9-4033-8b81-75bd2c812d2d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:44.757415 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f4385c-b0a9-4033-8b81-75bd2c812d2d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:44.759567 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f4385c-b0a9-4033-8b81-75bd2c812d2d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:44.773167 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:44.784714 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 344/681] 10.222.0.40 () {56 vars in 1135 bytes} [Mon Jul 18 06:06:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:44.785952 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4385c-b0a9-4033-8b81-75bd2c812d2d None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:44.795038 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d933c884-22c0-45ec-8f82-521e5fb7a536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:44.795871 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933c884-22c0-45ec-8f82-521e5fb7a536 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:44.796268 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933c884-22c0-45ec-8f82-521e5fb7a536 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:44.796859 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933c884-22c0-45ec-8f82-521e5fb7a536 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:44.798517 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d933c884-22c0-45ec-8f82-521e5fb7a536 None admin] RBAC: Adding query filter params (role.id=ab10744ff77646eab23a769ab89c74aa, scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a, user.id=9824a96e2bcd4a16bfc6fe6bf4ea1ad5) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:44.798944 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d933c884-22c0-45ec-8f82-521e5fb7a536 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:44.800939 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d933c884-22c0-45ec-8f82-521e5fb7a536 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:44.811462 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 338/682] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:06:44 2022] GET /identity/v3/role_assignments?user.id=9824a96e2bcd4a16bfc6fe6bf4ea1ad5&scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a&role.id=ab10744ff77646eab23a769ab89c74aa => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:44.815161 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d933c884-22c0-45ec-8f82-521e5fb7a536 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:44.999203 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4385c-b0a9-4033-8b81-75bd2c812d2d None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:45.024771 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:45.035943 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d933c884-22c0-45ec-8f82-521e5fb7a536 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:45.065468 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab1adf-fbad-4ec3-a227-1a28a6ca7830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:45.066615 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab1adf-fbad-4ec3-a227-1a28a6ca7830 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:45.067116 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab1adf-fbad-4ec3-a227-1a28a6ca7830 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:45.068084 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab1adf-fbad-4ec3-a227-1a28a6ca7830 None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:45.069217 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab1adf-fbad-4ec3-a227-1a28a6ca7830 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:45.069746 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab1adf-fbad-4ec3-a227-1a28a6ca7830 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:45.072820 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab1adf-fbad-4ec3-a227-1a28a6ca7830 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:45.088621 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 345/683] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 18 06:06:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:45.089984 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8aef9-21a8-4dbb-aeb0-7160d47c52ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:45.090681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8aef9-21a8-4dbb-aeb0-7160d47c52ee None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:45.090969 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8aef9-21a8-4dbb-aeb0-7160d47c52ee None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:45.092908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8aef9-21a8-4dbb-aeb0-7160d47c52ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:45.095081 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab1adf-fbad-4ec3-a227-1a28a6ca7830 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:45.180357 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:45.218068 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b8aef9-21a8-4dbb-aeb0-7160d47c52ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:45.219926 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b8aef9-21a8-4dbb-aeb0-7160d47c52ee None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:45.220259 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab1adf-fbad-4ec3-a227-1a28a6ca7830 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:45.258302 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c60197c-ced1-4b98-b724-81c81e2daf03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:45.259053 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60197c-ced1-4b98-b724-81c81e2daf03 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:45.259360 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60197c-ced1-4b98-b724-81c81e2daf03 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:45.259649 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60197c-ced1-4b98-b724-81c81e2daf03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:45.260666 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c60197c-ced1-4b98-b724-81c81e2daf03 None admin] RBAC: Adding query filter params (role.id=ab10744ff77646eab23a769ab89c74aa, scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a, user.id=1193330cb5644765a7abb77738929ba4) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:45.260993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c60197c-ced1-4b98-b724-81c81e2daf03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:45.262448 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c60197c-ced1-4b98-b724-81c81e2daf03 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:45.269496 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 346/684] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:06:45 2022] GET /identity/v3/role_assignments?user.id=1193330cb5644765a7abb77738929ba4&scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a&role.id=ab10744ff77646eab23a769ab89c74aa => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:45.354194 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943ef15-1fb6-4cf3-91aa-c7f96fc0a125 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:45.354536 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943ef15-1fb6-4cf3-91aa-c7f96fc0a125 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:45.354857 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943ef15-1fb6-4cf3-91aa-c7f96fc0a125 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:45.356252 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 347/685] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:06:45.362209 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73bcf7-28c2-46dd-be33-0cfb2548ec40 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:45.362497 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73bcf7-28c2-46dd-be33-0cfb2548ec40 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:45.362794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73bcf7-28c2-46dd-be33-0cfb2548ec40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:45.391320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-bf73bcf7-28c2-46dd-be33-0cfb2548ec40 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:45.392590 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-bf73bcf7-28c2-46dd-be33-0cfb2548ec40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:45.416192 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 339/686] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:06:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 06:06:45.422143 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8aef9-21a8-4dbb-aeb0-7160d47c52ee None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:45.442504 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:45.508097 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 348/687] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 06:06:45.511127 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721185a8-6da4-45bb-b7da-0809449b8264 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:45.511811 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721185a8-6da4-45bb-b7da-0809449b8264 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:45.512630 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721185a8-6da4-45bb-b7da-0809449b8264 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:45.518311 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 349/688] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:45.520856 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290cb2d-4ac7-4736-8a6b-01fdea986f08 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:45.521141 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290cb2d-4ac7-4736-8a6b-01fdea986f08 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:45.521416 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290cb2d-4ac7-4736-8a6b-01fdea986f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:45.541449 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-c290cb2d-4ac7-4736-8a6b-01fdea986f08 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:45.542731 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c290cb2d-4ac7-4736-8a6b-01fdea986f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:45.547237 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:45.591199 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8aef9-21a8-4dbb-aeb0-7160d47c52ee None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:45.637429 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 350/689] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 06:06:45.638557 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c290cb2d-4ac7-4736-8a6b-01fdea986f08 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:45.676992 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee120548-7c27-4b4f-b7c6-e69b8097c61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:45.678692 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee120548-7c27-4b4f-b7c6-e69b8097c61d None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:45.679406 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee120548-7c27-4b4f-b7c6-e69b8097c61d None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:45.680072 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee120548-7c27-4b4f-b7c6-e69b8097c61d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:45.704263 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee120548-7c27-4b4f-b7c6-e69b8097c61d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:45.705784 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee120548-7c27-4b4f-b7c6-e69b8097c61d None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:45.729487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-ee120548-7c27-4b4f-b7c6-e69b8097c61d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:45.729487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:45.729487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:45.729487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:45.729487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:45.729487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:45.729487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:45.729487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:45.729487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:45.729487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:45.729487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:45.729487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:45.729487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:45.729487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:45.729487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:45.730460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:45.730460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:45.730460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:45.730460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:45.730460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:45.730460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:45.730460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:45.730460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:45.730460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:45.730460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:45.730460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:45.730460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:45.730460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:45.730460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:45.730460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:45.730460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:45.730460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:45.732074 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:45.732074 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:45.732074 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:45.732074 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:45.732074 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 340/690] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:06:45.732074 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee120548-7c27-4b4f-b7c6-e69b8097c61d None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:45.762874 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:45.766416 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c290cb2d-4ac7-4736-8a6b-01fdea986f08 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:45.803566 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8486e96-8e39-403b-bf31-aea66966aa60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:45.804444 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8486e96-8e39-403b-bf31-aea66966aa60 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:45.804703 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8486e96-8e39-403b-bf31-aea66966aa60 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:45.804996 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8486e96-8e39-403b-bf31-aea66966aa60 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:45.817204 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8486e96-8e39-403b-bf31-aea66966aa60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:45.825230 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-e8486e96-8e39-403b-bf31-aea66966aa60 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:06:45.825230 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:45.825230 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:45.825230 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:45.825230 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:45.825230 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:45.825230 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:45.825230 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:45.825230 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:45.825230 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:45.825230 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:45.825230 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:45.825230 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:45.825230 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:45.825230 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:45.825230 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:45.826231 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:45.826231 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:45.826231 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:45.826231 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:45.826231 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:45.826231 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:45.826231 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:45.826231 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:45.826231 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:45.826231 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:45.826231 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:45.826231 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:45.826231 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:45.826231 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:45.826231 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:45.826231 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:45.827736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:45.827736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:45.827736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:45.827736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:45.827736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:45.827736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:45.827736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:45.827736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:45.827736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:45.827736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:45.827736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:06:45.827736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:45.827736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 351/691] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 18 06:06:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:45.827736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8486e96-8e39-403b-bf31-aea66966aa60 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:45.858698 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee120548-7c27-4b4f-b7c6-e69b8097c61d None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:45.900844 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae029f51-eb13-43ed-8ad8-2b500b5ce408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:45.902078 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae029f51-eb13-43ed-8ad8-2b500b5ce408 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:45.902686 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae029f51-eb13-43ed-8ad8-2b500b5ce408 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:45.903197 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae029f51-eb13-43ed-8ad8-2b500b5ce408 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:45.904172 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae029f51-eb13-43ed-8ad8-2b500b5ce408 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:45.905874 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae029f51-eb13-43ed-8ad8-2b500b5ce408 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:45.908232 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:45.923337 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-ae029f51-eb13-43ed-8ad8-2b500b5ce408 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 06:06:45.923337 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:45.923337 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 06:06:45.923337 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 06:06:45.923337 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 06:06:45.923337 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 06:06:45.923337 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 06:06:45.923337 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:06:45.923337 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 06:06:45.923337 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:45.923337 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 06:06:45.923337 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:45.923337 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:45.923337 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:45.923337 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:45.923337 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:45.924722 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:45.924722 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:45.924722 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:45.924722 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:45.924722 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:45.924722 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:45.924722 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:45.924722 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 06:06:45.924722 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 06:06:45.924722 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 06:06:45.924722 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 06:06:45.924722 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:45.924722 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:45.924722 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:45.924722 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:45.924722 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:45.925688 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:45.925688 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:45.925688 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:45.925688 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:45.925688 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:45.925688 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:45.925688 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:45.925688 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:45.925688 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:45.925688 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:45.925688 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:45.925688 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 06:06:45.925688 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:06:45.925688 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 06:06:45.925688 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:45.928365 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 341/692] 10.222.0.40 () {56 vars in 1129 bytes} [Mon Jul 18 06:06:45 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:45.929604 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae029f51-eb13-43ed-8ad8-2b500b5ce408 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:45.937867 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8486e96-8e39-403b-bf31-aea66966aa60 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:45.974949 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c92052-364d-4c99-83be-dee451791d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:45.975633 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c92052-364d-4c99-83be-dee451791d09 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:45.975920 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c92052-364d-4c99-83be-dee451791d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:45.976199 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c92052-364d-4c99-83be-dee451791d09 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:45.976969 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c92052-364d-4c99-83be-dee451791d09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:45.977278 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c92052-364d-4c99-83be-dee451791d09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:45.979374 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c92052-364d-4c99-83be-dee451791d09 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:45.992369 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 352/693] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:45.993271 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c92052-364d-4c99-83be-dee451791d09 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:46.013216 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:46.048460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae029f51-eb13-43ed-8ad8-2b500b5ce408 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:46.087533 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feef63d-f128-407a-9206-67f350758cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:46.088803 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feef63d-f128-407a-9206-67f350758cf4 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:46.089422 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feef63d-f128-407a-9206-67f350758cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:46.089902 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feef63d-f128-407a-9206-67f350758cf4 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:46.090906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feef63d-f128-407a-9206-67f350758cf4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:46.091424 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feef63d-f128-407a-9206-67f350758cf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:46.093309 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feef63d-f128-407a-9206-67f350758cf4 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:46.098916 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c92052-364d-4c99-83be-dee451791d09 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:46.104306 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 342/694] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:06:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:46.105608 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feef63d-f128-407a-9206-67f350758cf4 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:46.135761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74367700-e152-49e8-85ac-2ec6c63ecb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:46.136469 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74367700-e152-49e8-85ac-2ec6c63ecb04 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:46.136762 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74367700-e152-49e8-85ac-2ec6c63ecb04 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:46.137043 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74367700-e152-49e8-85ac-2ec6c63ecb04 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:46.137818 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74367700-e152-49e8-85ac-2ec6c63ecb04 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:46.138059 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74367700-e152-49e8-85ac-2ec6c63ecb04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:46.139449 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74367700-e152-49e8-85ac-2ec6c63ecb04 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:46.144444 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 353/695] 10.222.0.40 () {56 vars in 1124 bytes} [Mon Jul 18 06:06:45 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:46.145295 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74367700-e152-49e8-85ac-2ec6c63ecb04 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:46.196124 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:46.235325 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:46.247987 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feef63d-f128-407a-9206-67f350758cf4 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:46.266923 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74367700-e152-49e8-85ac-2ec6c63ecb04 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:46.291030 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b89989-4985-4b0e-8f07-3cabb626dec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:46.291868 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b89989-4985-4b0e-8f07-3cabb626dec2 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:46.292153 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b89989-4985-4b0e-8f07-3cabb626dec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:46.292443 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b89989-4985-4b0e-8f07-3cabb626dec2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:46.301018 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b89989-4985-4b0e-8f07-3cabb626dec2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:46.302923 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b89989-4985-4b0e-8f07-3cabb626dec2 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:46.316361 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb911c9-2287-473f-8212-8b3fb3de8c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:46.317586 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb911c9-2287-473f-8212-8b3fb3de8c8e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:46.318078 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb911c9-2287-473f-8212-8b3fb3de8c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:46.318581 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb911c9-2287-473f-8212-8b3fb3de8c8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:46.319447 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-34b89989-4985-4b0e-8f07-3cabb626dec2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:46.319447 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:46.319447 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:46.319447 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:46.319447 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:46.319447 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:46.319447 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:46.319447 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:46.319447 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:46.319447 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:46.319447 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:46.319447 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:46.319447 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 06:06:46.319447 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 06:06:46.319447 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 06:06:46.320332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 06:06:46.320332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:46.320332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:46.320332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:46.320332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:46.320332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:46.320332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:46.320332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:46.320332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:46.320332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:46.320332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:46.320332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:46.320332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:46.320332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:46.320332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:46.320332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:46.320332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:46.321874 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 06:06:46.321874 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 06:06:46.321874 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 06:06:46.321874 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 06:06:46.321874 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 06:06:46.321874 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 06:06:46.321874 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:46.321874 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:46.326126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 343/696] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:06:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:46.327231 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b89989-4985-4b0e-8f07-3cabb626dec2 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:46.340998 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb911c9-2287-473f-8212-8b3fb3de8c8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:46.343622 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb911c9-2287-473f-8212-8b3fb3de8c8e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:46.371503 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-eeb911c9-2287-473f-8212-8b3fb3de8c8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:46.371503 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:46.371503 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:46.371503 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:46.371503 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:46.371503 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:46.371503 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:46.371503 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:46.371503 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:46.371503 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:46.371503 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:46.371503 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:46.371503 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:46.371503 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:46.371503 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:46.372490 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:46.372490 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:46.372490 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:46.372490 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:46.372490 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:46.372490 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:46.372490 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:46.372490 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:46.372490 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:46.372490 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:46.372490 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:46.372490 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:46.372490 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:46.372490 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:46.372490 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:46.372490 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:46.372490 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:46.373391 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:46.373391 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:46.373391 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:46.373391 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:46.376330 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 354/697] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:46.378075 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb911c9-2287-473f-8212-8b3fb3de8c8e None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:46.462738 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b89989-4985-4b0e-8f07-3cabb626dec2 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:46.490071 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:46.503379 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb911c9-2287-473f-8212-8b3fb3de8c8e None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:46.510145 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a11d9-f681-488f-bbb2-2965df887f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:46.511132 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a11d9-f681-488f-bbb2-2965df887f9b None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:46.511627 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a11d9-f681-488f-bbb2-2965df887f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:46.512178 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a11d9-f681-488f-bbb2-2965df887f9b None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:46.513791 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585a11d9-f681-488f-bbb2-2965df887f9b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:46.514316 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585a11d9-f681-488f-bbb2-2965df887f9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:46.516378 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585a11d9-f681-488f-bbb2-2965df887f9b None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:46.532764 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 344/698] 10.222.0.40 () {56 vars in 1124 bytes} [Mon Jul 18 06:06:46 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:46.533733 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a11d9-f681-488f-bbb2-2965df887f9b None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:46.568085 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d3652-440b-40d7-8b54-9665695da89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:46.568835 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d3652-440b-40d7-8b54-9665695da89c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:46.569127 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d3652-440b-40d7-8b54-9665695da89c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:46.569404 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d3652-440b-40d7-8b54-9665695da89c None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:46.570351 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6d3652-440b-40d7-8b54-9665695da89c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:46.572287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6d3652-440b-40d7-8b54-9665695da89c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:46.575888 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6d3652-440b-40d7-8b54-9665695da89c None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:46.578044 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:46.605426 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 355/699] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 18 06:06:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:46.611653 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d3652-440b-40d7-8b54-9665695da89c None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:46.747663 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a11d9-f681-488f-bbb2-2965df887f9b None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:46.790920 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7550a580-8a21-46bc-9c25-cd0a3fe16c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:46.791901 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550a580-8a21-46bc-9c25-cd0a3fe16c51 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:46.792193 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550a580-8a21-46bc-9c25-cd0a3fe16c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:46.792505 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550a580-8a21-46bc-9c25-cd0a3fe16c51 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:46.793352 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7550a580-8a21-46bc-9c25-cd0a3fe16c51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:46.793551 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7550a580-8a21-46bc-9c25-cd0a3fe16c51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:46.795000 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7550a580-8a21-46bc-9c25-cd0a3fe16c51 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:46.802802 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d3652-440b-40d7-8b54-9665695da89c None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:46.808652 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 345/700] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:46.813811 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7550a580-8a21-46bc-9c25-cd0a3fe16c51 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:46.843792 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80857b6d-e070-492d-a2f6-b31e1f88f1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:46.844756 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80857b6d-e070-492d-a2f6-b31e1f88f1d7 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:46.845260 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80857b6d-e070-492d-a2f6-b31e1f88f1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:46.845690 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80857b6d-e070-492d-a2f6-b31e1f88f1d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:46.846791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80857b6d-e070-492d-a2f6-b31e1f88f1d7 None admin] RBAC: Adding query filter params (role.id=ab10744ff77646eab23a769ab89c74aa, scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a, user.id=0681f5fbea6d415c9ad2f3afae256a5c) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:46.847307 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80857b6d-e070-492d-a2f6-b31e1f88f1d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:46.849092 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80857b6d-e070-492d-a2f6-b31e1f88f1d7 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:46.857364 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 356/701] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:06:46 2022] GET /identity/v3/role_assignments?user.id=0681f5fbea6d415c9ad2f3afae256a5c&scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a&role.id=ab10744ff77646eab23a769ab89c74aa => generated 608 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:46.868482 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:46.957215 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7550a580-8a21-46bc-9c25-cd0a3fe16c51 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:47.006123 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd2b78-bc8a-4031-98c3-8359ee7b0365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:47.006944 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd2b78-bc8a-4031-98c3-8359ee7b0365 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:47.007233 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd2b78-bc8a-4031-98c3-8359ee7b0365 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:47.007568 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd2b78-bc8a-4031-98c3-8359ee7b0365 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:47.017406 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cd2b78-bc8a-4031-98c3-8359ee7b0365 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:47.018899 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cd2b78-bc8a-4031-98c3-8359ee7b0365 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:47.041908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-93cd2b78-bc8a-4031-98c3-8359ee7b0365 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 06:06:47.041908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:47.041908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:47.041908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:47.041908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:47.041908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:47.041908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:47.041908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:47.041908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:47.041908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:47.041908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:47.041908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:47.041908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 06:06:47.041908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 06:06:47.041908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 06:06:47.041908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 06:06:47.043035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:47.043035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:47.043035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 06:06:47.043035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:47.043035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 06:06:47.043035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:47.043035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:47.043035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:47.043035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:47.043035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:47.043035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:47.043035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:47.043035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:47.043035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:47.043035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:47.043035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:47.043035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:47.043035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:47.043996 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:47.043996 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:47.043996 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 06:06:47.043996 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 06:06:47.043996 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 06:06:47.043996 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 06:06:47.043996 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 06:06:47.043996 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 06:06:47.043996 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 06:06:47.043996 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:47.046353 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 346/702] 10.222.0.40 () {56 vars in 1093 bytes} [Mon Jul 18 06:06:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:47.051710 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80857b6d-e070-492d-a2f6-b31e1f88f1d7 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:47.122476 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:47.169139 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80857b6d-e070-492d-a2f6-b31e1f88f1d7 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:47.207923 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3a858-bffd-4054-8b8c-d61815a0e453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:47.208647 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3a858-bffd-4054-8b8c-d61815a0e453 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:47.208938 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3a858-bffd-4054-8b8c-d61815a0e453 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:47.209219 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3a858-bffd-4054-8b8c-d61815a0e453 None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:47.210110 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a3a858-bffd-4054-8b8c-d61815a0e453 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:47.210428 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a3a858-bffd-4054-8b8c-d61815a0e453 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:47.212579 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a3a858-bffd-4054-8b8c-d61815a0e453 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:47.233107 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 357/703] 10.222.0.40 () {56 vars in 1133 bytes} [Mon Jul 18 06:06:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:47.241565 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd2b78-bc8a-4031-98c3-8359ee7b0365 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:47.359138 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd2b78-bc8a-4031-98c3-8359ee7b0365 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:47.361716 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:47.401161 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bacf802-c600-4301-b8f5-63db4609f797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:47.401856 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacf802-c600-4301-b8f5-63db4609f797 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:47.402130 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacf802-c600-4301-b8f5-63db4609f797 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:47.402411 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacf802-c600-4301-b8f5-63db4609f797 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:47.431639 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5966933-8729-4ad2-b7d5-9f3b7bfef9fe None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:47.431985 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5966933-8729-4ad2-b7d5-9f3b7bfef9fe None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:47.432209 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5966933-8729-4ad2-b7d5-9f3b7bfef9fe None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:47.433883 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 358/704] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:47.439192 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6fb7e8-c50a-4702-ba24-b44d08425884 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:47.439488 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6fb7e8-c50a-4702-ba24-b44d08425884 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:47.439778 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6fb7e8-c50a-4702-ba24-b44d08425884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:47.477349 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-df6fb7e8-c50a-4702-ba24-b44d08425884 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:47.478934 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-df6fb7e8-c50a-4702-ba24-b44d08425884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:47.491789 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bacf802-c600-4301-b8f5-63db4609f797 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:47.494413 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bacf802-c600-4301-b8f5-63db4609f797 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:47.609847 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 359/705] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:06:47.633147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 347/706] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:06:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 06:06:47.641686 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6fb7e8-c50a-4702-ba24-b44d08425884 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:47.658468 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bacf802-c600-4301-b8f5-63db4609f797 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:47.667936 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:47.739811 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:47.810462 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bacf802-c600-4301-b8f5-63db4609f797 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:47.811469 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6fb7e8-c50a-4702-ba24-b44d08425884 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:47.852170 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0d355-db52-4780-a559-62fb6b9d9f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:47.853507 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0d355-db52-4780-a559-62fb6b9d9f2a None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:47.854008 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0d355-db52-4780-a559-62fb6b9d9f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:47.854142 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df07d243-c8fe-4c99-97bd-d920fa9c78cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:47.854663 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0d355-db52-4780-a559-62fb6b9d9f2a None admin] PATH_INFO: `/v3/services/network` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:47.854848 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df07d243-c8fe-4c99-97bd-d920fa9c78cd None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:47.855130 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df07d243-c8fe-4c99-97bd-d920fa9c78cd None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:47.855410 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df07d243-c8fe-4c99-97bd-d920fa9c78cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:47.856389 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb0d355-db52-4780-a559-62fb6b9d9f2a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:47.858867 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb0d355-db52-4780-a559-62fb6b9d9f2a None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:47.867525 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df07d243-c8fe-4c99-97bd-d920fa9c78cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:47.869080 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df07d243-c8fe-4c99-97bd-d920fa9c78cd None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:47.875488 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-3bb0d355-db52-4780-a559-62fb6b9d9f2a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 06:06:47.875488 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:47.875488 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 06:06:47.875488 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 06:06:47.875488 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 06:06:47.875488 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 06:06:47.875488 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 06:06:47.875488 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:06:47.875488 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 06:06:47.875488 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:47.875488 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 06:06:47.875488 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:47.875488 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:47.875488 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:47.875488 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:47.875488 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:47.875488 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:47.876613 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:47.876613 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:47.876613 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:47.876613 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:47.876613 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:47.876613 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:47.876613 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 06:06:47.876613 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 06:06:47.876613 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 06:06:47.876613 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 06:06:47.876613 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:47.876613 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:47.876613 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:47.876613 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:47.876613 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:47.876613 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:47.877536 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:47.877536 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:47.877536 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:47.877536 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:47.877536 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:47.877536 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:47.877536 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:47.877536 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:47.877536 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:47.877536 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:47.877536 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 06:06:47.877536 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:06:47.877536 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 06:06:47.877536 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:47.880559 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 348/707] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:06:47 2022] GET /identity/v3/services/network => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:47.882230 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0599f-ed6f-4247-b3ae-6d1189fecf75 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:47.882732 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0599f-ed6f-4247-b3ae-6d1189fecf75 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:47.883270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0599f-ed6f-4247-b3ae-6d1189fecf75 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:47.889837 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 349/708] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:47.889837 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf0599f-ed6f-4247-b3ae-6d1189fecf75 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:47.902386 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-df07d243-c8fe-4c99-97bd-d920fa9c78cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:47.902386 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:47.902386 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:47.902386 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:47.902386 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:47.902386 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:47.902386 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:47.902386 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:47.902386 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:47.902386 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:47.902386 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:47.902386 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:47.902386 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:47.902386 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:47.902386 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:47.903566 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:47.903566 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:47.903566 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:47.903566 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:47.903566 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:47.903566 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:47.903566 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:47.903566 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:47.903566 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:47.903566 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:47.903566 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:47.903566 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:47.903566 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:47.903566 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:47.903566 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:47.903566 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:47.903566 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:47.904801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:47.904801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:47.904801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:47.904801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:47.910944 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 360/709] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:47.910944 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28237fa0-1d97-4412-a804-17a07b76e2f1 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:47.910944 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28237fa0-1d97-4412-a804-17a07b76e2f1 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:47.910944 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28237fa0-1d97-4412-a804-17a07b76e2f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:47.933124 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-28237fa0-1d97-4412-a804-17a07b76e2f1 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:47.934757 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-28237fa0-1d97-4412-a804-17a07b76e2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:47.981486 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:48.018719 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf0599f-ed6f-4247-b3ae-6d1189fecf75 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:48.034500 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 361/710] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:06:48.034500 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28237fa0-1d97-4412-a804-17a07b76e2f1 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:48.083250 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:48.085029 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac65245b-d172-49b8-ad0a-80a3fe82e67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:48.086138 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65245b-d172-49b8-ad0a-80a3fe82e67c None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:48.087557 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65245b-d172-49b8-ad0a-80a3fe82e67c None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:48.088358 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65245b-d172-49b8-ad0a-80a3fe82e67c None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:48.089328 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac65245b-d172-49b8-ad0a-80a3fe82e67c None admin] RBAC: Adding query filter params (name=network) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:48.089826 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac65245b-d172-49b8-ad0a-80a3fe82e67c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:48.092636 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac65245b-d172-49b8-ad0a-80a3fe82e67c None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:48.098900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 350/711] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Jul 18 06:06:47 2022] GET /identity/v3/services?name=network => generated 124 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:48.100002 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac65245b-d172-49b8-ad0a-80a3fe82e67c None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:48.195225 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28237fa0-1d97-4412-a804-17a07b76e2f1 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:48.213798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac65245b-d172-49b8-ad0a-80a3fe82e67c None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:48.241135 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760cda24-cdb5-4ba6-943f-ae96f87d826b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:48.241892 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760cda24-cdb5-4ba6-943f-ae96f87d826b None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:48.242298 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760cda24-cdb5-4ba6-943f-ae96f87d826b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:48.242748 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760cda24-cdb5-4ba6-943f-ae96f87d826b None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:48.244133 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760cda24-cdb5-4ba6-943f-ae96f87d826b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:48.244539 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760cda24-cdb5-4ba6-943f-ae96f87d826b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:48.246042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760cda24-cdb5-4ba6-943f-ae96f87d826b None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:48.250826 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:48.257998 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90388be4-9a9e-42e8-ade8-ca5a40a76335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:48.263580 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 362/712] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:48.263580 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760cda24-cdb5-4ba6-943f-ae96f87d826b None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:48.263784 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90388be4-9a9e-42e8-ade8-ca5a40a76335 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:48.263784 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90388be4-9a9e-42e8-ade8-ca5a40a76335 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:48.263784 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90388be4-9a9e-42e8-ade8-ca5a40a76335 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:48.276584 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90388be4-9a9e-42e8-ade8-ca5a40a76335 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:48.278351 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90388be4-9a9e-42e8-ade8-ca5a40a76335 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:48.303914 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-90388be4-9a9e-42e8-ade8-ca5a40a76335 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:48.303914 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:48.303914 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:48.303914 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:48.303914 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:48.303914 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:48.303914 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:48.303914 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:48.303914 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:48.303914 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:48.303914 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:48.303914 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:48.303914 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:48.303914 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:48.303914 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:48.304924 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:48.304924 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:48.304924 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:48.304924 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:48.304924 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:48.304924 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:48.304924 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:48.304924 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:48.304924 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:48.304924 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:48.304924 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:48.304924 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:48.304924 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:48.304924 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:48.304924 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:48.304924 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:48.304924 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:48.306453 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:48.306453 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:48.306453 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:48.306453 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:48.316248 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 351/713] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:48.317121 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90388be4-9a9e-42e8-ade8-ca5a40a76335 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:48.386925 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:48.399908 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760cda24-cdb5-4ba6-943f-ae96f87d826b None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:48.436131 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b76d9e8-9071-46ea-8e94-cb903a683e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:48.436872 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76d9e8-9071-46ea-8e94-cb903a683e11 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:48.437152 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76d9e8-9071-46ea-8e94-cb903a683e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:48.437428 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76d9e8-9071-46ea-8e94-cb903a683e11 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:48.438403 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b76d9e8-9071-46ea-8e94-cb903a683e11 None admin] RBAC: Adding query filter params (type=network) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:48.438758 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b76d9e8-9071-46ea-8e94-cb903a683e11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:48.440313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b76d9e8-9071-46ea-8e94-cb903a683e11 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:48.452867 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 363/714] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Jul 18 06:06:48 2022] GET /identity/v3/services?type=network => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:48.453731 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b76d9e8-9071-46ea-8e94-cb903a683e11 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:48.462188 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90388be4-9a9e-42e8-ade8-ca5a40a76335 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:48.517105 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e87f928-9397-4a91-8810-51d38f7d7aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:48.517969 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e87f928-9397-4a91-8810-51d38f7d7aee None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:48.518285 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e87f928-9397-4a91-8810-51d38f7d7aee None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:48.518587 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e87f928-9397-4a91-8810-51d38f7d7aee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:48.541801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e87f928-9397-4a91-8810-51d38f7d7aee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:48.543247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e87f928-9397-4a91-8810-51d38f7d7aee None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:48.576195 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-8e87f928-9397-4a91-8810-51d38f7d7aee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:48.576195 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:48.576195 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:48.576195 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:48.576195 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:48.576195 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:48.576195 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:48.576195 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:48.576195 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:48.576195 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:48.576195 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:48.576195 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:48.576195 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 06:06:48.576195 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 06:06:48.576195 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 06:06:48.577227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 06:06:48.577227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:48.577227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:48.577227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:48.577227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:48.577227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:48.577227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:48.577227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:48.577227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:48.577227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:48.577227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:48.577227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:48.577227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:48.577227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:48.577227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:48.577227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:48.577227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:48.578203 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 06:06:48.578203 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 06:06:48.578203 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 06:06:48.578203 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 06:06:48.578203 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 06:06:48.578203 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 06:06:48.578203 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:48.578203 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:48.578203 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 352/715] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:06:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:48.586670 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e87f928-9397-4a91-8810-51d38f7d7aee None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:48.696853 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:48.711859 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b76d9e8-9071-46ea-8e94-cb903a683e11 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:48.749645 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b6c9c-2a53-43fc-8d58-fb729de33e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:48.750712 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b6c9c-2a53-43fc-8d58-fb729de33e31 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:48.751227 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b6c9c-2a53-43fc-8d58-fb729de33e31 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:48.751729 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b6c9c-2a53-43fc-8d58-fb729de33e31 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:48.752730 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870b6c9c-2a53-43fc-8d58-fb729de33e31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:48.753270 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870b6c9c-2a53-43fc-8d58-fb729de33e31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:48.754917 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870b6c9c-2a53-43fc-8d58-fb729de33e31 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:48.760361 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e87f928-9397-4a91-8810-51d38f7d7aee None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:48.772427 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 364/716] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:48.782206 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b6c9c-2a53-43fc-8d58-fb729de33e31 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:48.796150 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:48.802115 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7fbcc-c81f-4edf-a594-26138411371b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:48.802809 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7fbcc-c81f-4edf-a594-26138411371b None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:48.803092 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7fbcc-c81f-4edf-a594-26138411371b None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:48.803677 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7fbcc-c81f-4edf-a594-26138411371b None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:48.804521 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7fbcc-c81f-4edf-a594-26138411371b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:48.804855 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7fbcc-c81f-4edf-a594-26138411371b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:48.806732 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7fbcc-c81f-4edf-a594-26138411371b None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:48.823475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 353/717] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 18 06:06:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:48.833784 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7fbcc-c81f-4edf-a594-26138411371b None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:48.970331 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7fbcc-c81f-4edf-a594-26138411371b None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:48.975190 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:48.995148 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b6c9c-2a53-43fc-8d58-fb729de33e31 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:49.031604 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb1082-0eb8-4894-ac1e-95ffb335cf80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:49.032261 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb1082-0eb8-4894-ac1e-95ffb335cf80 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:49.032593 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb1082-0eb8-4894-ac1e-95ffb335cf80 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:49.032869 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb1082-0eb8-4894-ac1e-95ffb335cf80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:49.033700 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb1082-0eb8-4894-ac1e-95ffb335cf80 None admin] RBAC: Adding query filter params (role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc, scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a, user.id=f92d948810ea4241810473506b654f7a) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:49.034012 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb1082-0eb8-4894-ac1e-95ffb335cf80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:49.035562 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb1082-0eb8-4894-ac1e-95ffb335cf80 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:49.045745 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 354/718] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:06:48 2022] GET /identity/v3/role_assignments?user.id=f92d948810ea4241810473506b654f7a&scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a&role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:49.047126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734c4a6-1e1e-4a18-bb36-8f693a8d55ca None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:49.047421 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734c4a6-1e1e-4a18-bb36-8f693a8d55ca None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:49.047701 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734c4a6-1e1e-4a18-bb36-8f693a8d55ca None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:49.049363 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 355/719] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:49.054087 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51866efb-a2ac-406a-8031-3f815d6950cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:49.054787 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51866efb-a2ac-406a-8031-3f815d6950cf None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:49.055071 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51866efb-a2ac-406a-8031-3f815d6950cf None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:49.055349 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51866efb-a2ac-406a-8031-3f815d6950cf None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:49.056722 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51866efb-a2ac-406a-8031-3f815d6950cf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:49.058221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51866efb-a2ac-406a-8031-3f815d6950cf None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:49.058547 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157635b8-46f1-43c9-9e9f-3aeef65e31db None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:49.059039 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157635b8-46f1-43c9-9e9f-3aeef65e31db None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:49.059519 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157635b8-46f1-43c9-9e9f-3aeef65e31db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:49.081681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-157635b8-46f1-43c9-9e9f-3aeef65e31db None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:49.083543 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-157635b8-46f1-43c9-9e9f-3aeef65e31db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:49.164471 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 365/720] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Jul 18 06:06:48 2022] POST /identity/v3/users => generated 254 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:06:49.196031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 356/721] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:06:49.236755 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51866efb-a2ac-406a-8031-3f815d6950cf None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:49.319785 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:49.363860 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51866efb-a2ac-406a-8031-3f815d6950cf None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:49.417292 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1c908-d13c-43ff-b352-b5c0f7c91e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:49.418378 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1c908-d13c-43ff-b352-b5c0f7c91e4b None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:49.418878 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1c908-d13c-43ff-b352-b5c0f7c91e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:49.419356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1c908-d13c-43ff-b352-b5c0f7c91e4b None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:49.420306 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a1c908-d13c-43ff-b352-b5c0f7c91e4b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:49.421916 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a1c908-d13c-43ff-b352-b5c0f7c91e4b None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:49.533592 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 366/722] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Jul 18 06:06:49 2022] POST /identity/v3/services => generated 251 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:06:49.570444 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313558c-e19d-4b2e-9001-25979261207e None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:49.571146 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313558c-e19d-4b2e-9001-25979261207e None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:49.571146 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313558c-e19d-4b2e-9001-25979261207e None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:49.573312 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 357/723] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:49.579679 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffca0ab-4fc3-43c7-8a49-18c990c7d7fd None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:49.580052 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffca0ab-4fc3-43c7-8a49-18c990c7d7fd None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:49.580248 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffca0ab-4fc3-43c7-8a49-18c990c7d7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:49.599566 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-6ffca0ab-4fc3-43c7-8a49-18c990c7d7fd None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:49.600864 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-6ffca0ab-4fc3-43c7-8a49-18c990c7d7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:49.736846 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 367/724] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:49 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 06:06:49.773416 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9313558c-e19d-4b2e-9001-25979261207e None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:49.904241 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:49.910186 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9313558c-e19d-4b2e-9001-25979261207e None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:49.954613 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b71e27-3842-4700-8b7f-ff7446fb6876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:49.955441 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b71e27-3842-4700-8b7f-ff7446fb6876 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:49.955723 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b71e27-3842-4700-8b7f-ff7446fb6876 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:49.955999 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b71e27-3842-4700-8b7f-ff7446fb6876 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:49.975529 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b71e27-3842-4700-8b7f-ff7446fb6876 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:49.990449 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-40b71e27-3842-4700-8b7f-ff7446fb6876 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:06:49.990449 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:49.990449 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:49.990449 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:49.990449 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:49.990449 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:49.990449 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:49.990449 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:49.990449 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:49.990449 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:49.990449 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:49.990449 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:49.990449 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:49.990449 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:49.990449 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:49.990449 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:49.991463 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:49.991463 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:49.991463 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:49.991463 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:49.991463 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:49.991463 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:49.991463 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:49.991463 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:49.991463 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:49.991463 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:49.991463 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:49.991463 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:49.991463 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:49.991463 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:49.991463 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:49.991463 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:49.992325 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:49.992325 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:49.992325 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:49.992325 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:49.992325 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:49.992325 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:49.992325 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:49.992325 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:49.992325 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:49.992325 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:49.992325 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:06:49.992325 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:49.994396 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 358/725] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 18 06:06:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:50.001473 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffca0ab-4fc3-43c7-8a49-18c990c7d7fd None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:50.065143 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:50.121435 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffca0ab-4fc3-43c7-8a49-18c990c7d7fd None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:50.170943 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4710e6-7b42-4227-86bc-6f70deb08620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:50.171552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4710e6-7b42-4227-86bc-6f70deb08620 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:50.171821 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4710e6-7b42-4227-86bc-6f70deb08620 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:50.172107 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4710e6-7b42-4227-86bc-6f70deb08620 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:50.172943 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4710e6-7b42-4227-86bc-6f70deb08620 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:50.173256 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4710e6-7b42-4227-86bc-6f70deb08620 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:50.174661 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4710e6-7b42-4227-86bc-6f70deb08620 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:50.187952 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 368/726] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:06:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:50.193375 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b71e27-3842-4700-8b7f-ff7446fb6876 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:50.301179 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b71e27-3842-4700-8b7f-ff7446fb6876 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:50.337710 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272acef4-7bb7-455a-8f9f-db1c7fecfab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:50.338759 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272acef4-7bb7-455a-8f9f-db1c7fecfab9 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:50.339051 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272acef4-7bb7-455a-8f9f-db1c7fecfab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:50.339335 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272acef4-7bb7-455a-8f9f-db1c7fecfab9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:50.351504 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272acef4-7bb7-455a-8f9f-db1c7fecfab9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:50.353660 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272acef4-7bb7-455a-8f9f-db1c7fecfab9 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:50.372516 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-272acef4-7bb7-455a-8f9f-db1c7fecfab9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:50.372516 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:50.372516 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:50.372516 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:50.372516 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:50.372516 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:50.372516 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:50.372516 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:50.372516 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:50.372516 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:50.372516 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:50.372516 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:50.372516 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:50.372516 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:50.372516 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:50.373475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:50.373475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:50.373475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:50.373475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:50.373475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:50.373475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:50.373475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:50.373475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:50.373475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:50.373475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:50.373475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:50.373475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:50.373475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:50.373475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:50.373475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:50.373475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:50.373475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:50.374595 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:50.374595 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:50.374595 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:50.374595 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:50.374595 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 359/727] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:50.381589 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4710e6-7b42-4227-86bc-6f70deb08620 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:50.538296 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4710e6-7b42-4227-86bc-6f70deb08620 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:50.555800 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:50.584539 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1fb1a3-3d30-4cfb-919d-0faedfdb199e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:50.585276 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1fb1a3-3d30-4cfb-919d-0faedfdb199e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:50.585572 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1fb1a3-3d30-4cfb-919d-0faedfdb199e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:50.585856 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1fb1a3-3d30-4cfb-919d-0faedfdb199e None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:50.586598 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1fb1a3-3d30-4cfb-919d-0faedfdb199e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:50.587473 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1fb1a3-3d30-4cfb-919d-0faedfdb199e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:50.589532 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1fb1a3-3d30-4cfb-919d-0faedfdb199e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:50.607113 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 369/728] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:50.613518 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272acef4-7bb7-455a-8f9f-db1c7fecfab9 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:50.662524 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:50.735188 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272acef4-7bb7-455a-8f9f-db1c7fecfab9 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:50.772956 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482fbf9-b63a-40a5-8ee2-0ac14d89fdf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:50.773791 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482fbf9-b63a-40a5-8ee2-0ac14d89fdf1 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:50.774081 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482fbf9-b63a-40a5-8ee2-0ac14d89fdf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:50.774920 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482fbf9-b63a-40a5-8ee2-0ac14d89fdf1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:50.783411 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482fbf9-b63a-40a5-8ee2-0ac14d89fdf1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:50.784920 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482fbf9-b63a-40a5-8ee2-0ac14d89fdf1 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:50.801332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-7482fbf9-b63a-40a5-8ee2-0ac14d89fdf1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 06:06:50.801332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:50.801332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:50.801332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:50.801332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:50.801332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:50.801332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:50.801332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:50.801332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:50.801332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:50.801332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:50.801332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:50.801332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 06:06:50.801332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 06:06:50.801332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 06:06:50.801332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 06:06:50.802453 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:50.802453 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:50.802453 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 06:06:50.802453 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:50.802453 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 06:06:50.802453 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:50.802453 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:50.802453 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:50.802453 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:50.802453 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:50.802453 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:50.802453 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:50.802453 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:50.802453 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:50.802453 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:50.802453 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:50.802453 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:50.802453 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:50.803431 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:50.803431 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:50.803431 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 06:06:50.803431 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 06:06:50.803431 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 06:06:50.803431 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 06:06:50.803431 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 06:06:50.803431 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 06:06:50.803431 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 06:06:50.803431 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:50.803431 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 360/729] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 18 06:06:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:50.812183 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1fb1a3-3d30-4cfb-919d-0faedfdb199e None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:50.989238 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1fb1a3-3d30-4cfb-919d-0faedfdb199e None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:51.025145 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc09135c-ca84-41f8-ae24-313d609089fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:51.026105 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09135c-ca84-41f8-ae24-313d609089fe None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:51.026391 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09135c-ca84-41f8-ae24-313d609089fe None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:51.026671 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09135c-ca84-41f8-ae24-313d609089fe None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:51.027524 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc09135c-ca84-41f8-ae24-313d609089fe None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:51.027837 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc09135c-ca84-41f8-ae24-313d609089fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:51.029317 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc09135c-ca84-41f8-ae24-313d609089fe None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:51.060390 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 370/730] 10.222.0.40 () {56 vars in 1135 bytes} [Mon Jul 18 06:06:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:51.065930 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482fbf9-b63a-40a5-8ee2-0ac14d89fdf1 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:51.113979 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c8840c-3ceb-490a-92b5-afd66f8c702a None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:51.114326 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c8840c-3ceb-490a-92b5-afd66f8c702a None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:51.114544 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c8840c-3ceb-490a-92b5-afd66f8c702a None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:51.116272 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 371/731] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:51.134772 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf245aa2-334c-428c-81e0-2f21b4d05d30 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:51.135100 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf245aa2-334c-428c-81e0-2f21b4d05d30 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:51.135383 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf245aa2-334c-428c-81e0-2f21b4d05d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:51.173516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-bf245aa2-334c-428c-81e0-2f21b4d05d30 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:51.174623 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:51.174736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-bf245aa2-334c-428c-81e0-2f21b4d05d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:51.208298 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482fbf9-b63a-40a5-8ee2-0ac14d89fdf1 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:51.232796 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:51.278733 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843139af-b238-4968-96de-d49386984b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:51.279486 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843139af-b238-4968-96de-d49386984b71 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:51.279774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843139af-b238-4968-96de-d49386984b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:51.280059 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843139af-b238-4968-96de-d49386984b71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:51.317941 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 372/732] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:51 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 06:06:51.374342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843139af-b238-4968-96de-d49386984b71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:51.375844 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843139af-b238-4968-96de-d49386984b71 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:51.381484 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf245aa2-334c-428c-81e0-2f21b4d05d30 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:51.452230 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:51.485782 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 361/733] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:06:51 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 06:06:51.492223 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843139af-b238-4968-96de-d49386984b71 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:51.592858 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf245aa2-334c-428c-81e0-2f21b4d05d30 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:51.625925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843139af-b238-4968-96de-d49386984b71 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:51.639128 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76115f8-9307-496b-9a7e-f403d8d6c36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:51.639809 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76115f8-9307-496b-9a7e-f403d8d6c36e None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:51.640094 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76115f8-9307-496b-9a7e-f403d8d6c36e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:51.640370 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76115f8-9307-496b-9a7e-f403d8d6c36e None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:51.641207 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76115f8-9307-496b-9a7e-f403d8d6c36e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:51.642796 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76115f8-9307-496b-9a7e-f403d8d6c36e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:51.663594 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f4c30-cacc-4057-99a3-c0fdb08c4768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:51.664400 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f4c30-cacc-4057-99a3-c0fdb08c4768 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:51.664747 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f4c30-cacc-4057-99a3-c0fdb08c4768 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:51.665060 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f4c30-cacc-4057-99a3-c0fdb08c4768 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:51.676005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f4c30-cacc-4057-99a3-c0fdb08c4768 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:51.677486 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f4c30-cacc-4057-99a3-c0fdb08c4768 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:51.697391 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-df6f4c30-cacc-4057-99a3-c0fdb08c4768 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:51.697391 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:51.697391 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:51.697391 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:51.697391 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:51.697391 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:51.697391 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:51.697391 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:51.697391 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:51.697391 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:51.697391 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:51.697391 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:51.697391 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:51.697391 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:51.697391 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:51.698335 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:51.698335 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:51.698335 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:51.698335 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:51.698335 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:51.698335 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:51.698335 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:51.698335 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:51.698335 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:51.698335 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:51.698335 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:51.698335 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:51.698335 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:51.698335 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:51.698335 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:51.698335 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:51.698335 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:51.699247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:51.699247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:51.699247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:51.699247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:51.699247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 362/734] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:51.705447 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f4c30-cacc-4057-99a3-c0fdb08c4768 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:51.734855 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:51.739483 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 373/735] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Jul 18 06:06:51 2022] POST /identity/v3/services => generated 240 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:06:51.741174 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a5c38-33d9-45a3-9115-2260332d8245 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:51.741755 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a5c38-33d9-45a3-9115-2260332d8245 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:51.742260 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a5c38-33d9-45a3-9115-2260332d8245 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:51.744857 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 374/736] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:51.756298 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b28a2-d0fc-4dbd-9192-9813de5f7a33 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:51.757081 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b28a2-d0fc-4dbd-9192-9813de5f7a33 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:51.757561 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b28a2-d0fc-4dbd-9192-9813de5f7a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:51.795260 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-826b28a2-d0fc-4dbd-9192-9813de5f7a33 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:51.798688 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-826b28a2-d0fc-4dbd-9192-9813de5f7a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:51.856843 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:51.908642 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f4c30-cacc-4057-99a3-c0fdb08c4768 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:51.940002 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 375/737] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:06:51.941704 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223427f1-8d3d-4d40-891a-69095957b676 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:51.941704 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223427f1-8d3d-4d40-891a-69095957b676 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:51.941883 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223427f1-8d3d-4d40-891a-69095957b676 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:51.948173 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 376/738] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:51.949626 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b72e8-a80e-46df-bf0c-945c1d1bea41 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:51.949923 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b72e8-a80e-46df-bf0c-945c1d1bea41 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:51.950213 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b72e8-a80e-46df-bf0c-945c1d1bea41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:51.959224 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f536f28-3ef2-44db-904f-66d574b088d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:51.960213 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f536f28-3ef2-44db-904f-66d574b088d3 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:51.960781 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f536f28-3ef2-44db-904f-66d574b088d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:51.961431 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f536f28-3ef2-44db-904f-66d574b088d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:51.962958 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f536f28-3ef2-44db-904f-66d574b088d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:51.963368 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f536f28-3ef2-44db-904f-66d574b088d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:51.964873 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f536f28-3ef2-44db-904f-66d574b088d3 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:51.969064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-c31b72e8-a80e-46df-bf0c-945c1d1bea41 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:51.970503 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c31b72e8-a80e-46df-bf0c-945c1d1bea41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:51.977894 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 363/739] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:51.978757 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f536f28-3ef2-44db-904f-66d574b088d3 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:52.057661 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 377/740] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:06:52.060521 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31b72e8-a80e-46df-bf0c-945c1d1bea41 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:52.096059 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:52.123327 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f536f28-3ef2-44db-904f-66d574b088d3 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:52.174428 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bdfc9a-3d19-411a-ae1e-0a1f3d5ca118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:52.175343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdfc9a-3d19-411a-ae1e-0a1f3d5ca118 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:52.175701 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdfc9a-3d19-411a-ae1e-0a1f3d5ca118 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:52.176049 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdfc9a-3d19-411a-ae1e-0a1f3d5ca118 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:52.189279 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bdfc9a-3d19-411a-ae1e-0a1f3d5ca118 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:52.196626 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:52.197892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-a9bdfc9a-3d19-411a-ae1e-0a1f3d5ca118 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:06:52.197892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:52.197892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:52.197892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:52.197892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:52.197892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:52.197892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:52.197892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:52.197892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:52.197892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:52.197892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:52.197892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:52.197892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:52.197892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:52.197892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:52.197892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:52.198825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:52.198825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:52.198825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:52.198825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:52.198825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:52.198825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:52.198825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:52.198825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:52.198825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:52.198825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:52.198825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:52.198825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:52.198825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:52.198825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:52.198825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:52.198825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:52.199681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:52.199681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:52.199681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:52.199681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:52.199681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:52.199681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:52.199681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:52.199681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:52.199681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:52.199681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:52.199681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:06:52.199681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:52.204273 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 364/741] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 18 06:06:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:52.205345 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bdfc9a-3d19-411a-ae1e-0a1f3d5ca118 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:52.227002 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31b72e8-a80e-46df-bf0c-945c1d1bea41 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:52.280600 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430ce62-d7e2-4dff-98f6-3b030be6cec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:52.281723 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430ce62-d7e2-4dff-98f6-3b030be6cec8 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:52.282361 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430ce62-d7e2-4dff-98f6-3b030be6cec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:52.283656 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430ce62-d7e2-4dff-98f6-3b030be6cec8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:52.294035 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8430ce62-d7e2-4dff-98f6-3b030be6cec8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:52.296669 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8430ce62-d7e2-4dff-98f6-3b030be6cec8 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:52.316347 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-8430ce62-d7e2-4dff-98f6-3b030be6cec8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:52.316347 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:52.316347 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:52.316347 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:52.316347 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:52.316347 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:52.316347 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:52.316347 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:52.316347 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:52.316347 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:52.316347 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:52.316347 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:52.316347 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 06:06:52.316347 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 06:06:52.316347 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 06:06:52.317426 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 06:06:52.317426 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:52.317426 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:52.317426 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:52.317426 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:52.317426 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:52.317426 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:52.317426 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:52.317426 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:52.317426 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:52.317426 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:52.317426 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:52.317426 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:52.317426 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:52.317426 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:52.317426 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:52.317426 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:52.318310 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 06:06:52.318310 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 06:06:52.318310 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 06:06:52.318310 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 06:06:52.318310 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 06:06:52.318310 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 06:06:52.318310 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:52.318310 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:52.318740 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bdfc9a-3d19-411a-ae1e-0a1f3d5ca118 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:52.320428 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 378/742] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:06:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:52.321705 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430ce62-d7e2-4dff-98f6-3b030be6cec8 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:52.369155 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f23107-1999-4b15-b5dd-65fc35cb1f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:52.370908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f23107-1999-4b15-b5dd-65fc35cb1f57 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:52.371300 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f23107-1999-4b15-b5dd-65fc35cb1f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:52.371679 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f23107-1999-4b15-b5dd-65fc35cb1f57 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:52.388055 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f23107-1999-4b15-b5dd-65fc35cb1f57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:52.399003 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-b0f23107-1999-4b15-b5dd-65fc35cb1f57 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:06:52.399003 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:52.399003 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:52.399003 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:52.399003 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:52.399003 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:52.399003 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:52.399003 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:52.399003 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:52.399003 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:52.399003 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:52.399003 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:52.399003 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:52.399003 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:52.399003 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:52.399003 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:52.400223 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:52.400223 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:52.400223 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:52.400223 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:52.400223 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:52.400223 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:52.400223 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:52.400223 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:52.400223 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:52.400223 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:52.400223 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:52.400223 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:52.400223 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:52.400223 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:52.400223 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:52.400223 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:52.401126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:52.401126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:52.401126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:52.401126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:52.401126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:52.401126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:52.401126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:52.401126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:52.401126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:52.401126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:52.401126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:06:52.401126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:52.401775 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 365/743] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 18 06:06:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:52.402844 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f23107-1999-4b15-b5dd-65fc35cb1f57 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:52.408447 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:52.456347 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430ce62-d7e2-4dff-98f6-3b030be6cec8 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:52.496861 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3ecf1-6510-48e4-8616-b88f62ae07ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:52.498590 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3ecf1-6510-48e4-8616-b88f62ae07ae None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:52.498590 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3ecf1-6510-48e4-8616-b88f62ae07ae None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:52.498590 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3ecf1-6510-48e4-8616-b88f62ae07ae None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:52.499549 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec3ecf1-6510-48e4-8616-b88f62ae07ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:52.499912 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec3ecf1-6510-48e4-8616-b88f62ae07ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:52.501390 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec3ecf1-6510-48e4-8616-b88f62ae07ae None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:52.515326 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 379/744] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:06:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:52.516190 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3ecf1-6510-48e4-8616-b88f62ae07ae None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:52.527207 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:52.535198 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f23107-1999-4b15-b5dd-65fc35cb1f57 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:52.576277 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6644b9-0b9d-4d2c-8c9f-b9ebccd72d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:52.577191 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6644b9-0b9d-4d2c-8c9f-b9ebccd72d2d None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:52.577492 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6644b9-0b9d-4d2c-8c9f-b9ebccd72d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:52.578172 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6644b9-0b9d-4d2c-8c9f-b9ebccd72d2d None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:52.578996 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6644b9-0b9d-4d2c-8c9f-b9ebccd72d2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:52.579316 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6644b9-0b9d-4d2c-8c9f-b9ebccd72d2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:52.580802 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6644b9-0b9d-4d2c-8c9f-b9ebccd72d2d None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:52.594090 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 366/745] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 18 06:06:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:52.595227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fced826-01c9-4c48-996a-74223be23e97 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:52.595520 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fced826-01c9-4c48-996a-74223be23e97 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:52.595801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fced826-01c9-4c48-996a-74223be23e97 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:52.597206 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 367/746] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:52.601907 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fced826-01c9-4c48-996a-74223be23e97 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:52.652010 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3ecf1-6510-48e4-8616-b88f62ae07ae None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:52.671802 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:52.697405 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cea543-3a8c-4ce2-b25a-a4c03e6a6bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:52.698086 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cea543-3a8c-4ce2-b25a-a4c03e6a6bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:52.698506 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cea543-3a8c-4ce2-b25a-a4c03e6a6bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:52.698690 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cea543-3a8c-4ce2-b25a-a4c03e6a6bd0 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:52.699390 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cea543-3a8c-4ce2-b25a-a4c03e6a6bd0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:52.699697 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cea543-3a8c-4ce2-b25a-a4c03e6a6bd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:52.701249 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cea543-3a8c-4ce2-b25a-a4c03e6a6bd0 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:52.712890 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 380/747] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 18 06:06:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:52.714411 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bce484-a5b7-4ed8-a75e-ec8614672d06 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:52.714707 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bce484-a5b7-4ed8-a75e-ec8614672d06 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:52.715188 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bce484-a5b7-4ed8-a75e-ec8614672d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:52.724698 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fced826-01c9-4c48-996a-74223be23e97 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:52.735498 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-39bce484-a5b7-4ed8-a75e-ec8614672d06 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:52.736760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-39bce484-a5b7-4ed8-a75e-ec8614672d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:52.773345 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675bfc4-c3f2-4ebc-a4e3-452f2f40cdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:52.774227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675bfc4-c3f2-4ebc-a4e3-452f2f40cdb3 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:52.774528 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675bfc4-c3f2-4ebc-a4e3-452f2f40cdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:52.774807 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675bfc4-c3f2-4ebc-a4e3-452f2f40cdb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:52.787438 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1675bfc4-c3f2-4ebc-a4e3-452f2f40cdb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:52.789437 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1675bfc4-c3f2-4ebc-a4e3-452f2f40cdb3 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:52.811189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-1675bfc4-c3f2-4ebc-a4e3-452f2f40cdb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:52.811189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:52.811189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:52.811189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:52.811189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:52.811189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:52.811189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:52.811189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:52.811189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:52.811189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:52.811189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:52.811189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:52.811189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:52.811189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:52.811189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:52.812280 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:52.812280 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:52.812280 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:52.812280 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:52.812280 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:52.812280 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:52.812280 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:52.812280 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:52.812280 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:52.812280 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:52.812280 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:52.812280 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:52.812280 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:52.812280 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:52.812280 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:52.812280 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:52.812280 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:52.813328 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:52.813328 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:52.813328 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:52.813328 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:52.813328 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 368/748] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:52.813328 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675bfc4-c3f2-4ebc-a4e3-452f2f40cdb3 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:52.822331 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:52.833731 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 381/749] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:06:52.834487 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bce484-a5b7-4ed8-a75e-ec8614672d06 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:52.948226 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675bfc4-c3f2-4ebc-a4e3-452f2f40cdb3 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:52.963434 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bce484-a5b7-4ed8-a75e-ec8614672d06 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:52.965313 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:52.990048 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a6616-f203-40e8-99ac-39c8f4380d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:52.990996 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a6616-f203-40e8-99ac-39c8f4380d40 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:52.991390 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a6616-f203-40e8-99ac-39c8f4380d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:52.991773 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a6616-f203-40e8-99ac-39c8f4380d40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:52.992684 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a6616-f203-40e8-99ac-39c8f4380d40 None admin] RBAC: Adding query filter params (role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc, scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a, user.id=0681f5fbea6d415c9ad2f3afae256a5c) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:52.993631 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a6616-f203-40e8-99ac-39c8f4380d40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:52.995685 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a6616-f203-40e8-99ac-39c8f4380d40 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:53.002977 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 369/750] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:06:52 2022] GET /identity/v3/role_assignments?user.id=0681f5fbea6d415c9ad2f3afae256a5c&scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a&role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:53.003846 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a6616-f203-40e8-99ac-39c8f4380d40 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:53.014748 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb98883-5dac-4655-98cb-ccc8930e9e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:53.015704 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb98883-5dac-4655-98cb-ccc8930e9e9e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:53.016058 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb98883-5dac-4655-98cb-ccc8930e9e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:53.016449 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb98883-5dac-4655-98cb-ccc8930e9e9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:53.036763 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb98883-5dac-4655-98cb-ccc8930e9e9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:53.039968 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb98883-5dac-4655-98cb-ccc8930e9e9e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:53.077126 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-cbb98883-5dac-4655-98cb-ccc8930e9e9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:53.077126 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:53.077126 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:53.077126 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:53.077126 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:53.077126 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:53.077126 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:53.077126 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:53.077126 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:53.077126 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:53.077126 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:53.077126 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:53.077126 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:53.077126 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:53.077126 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:53.078132 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:53.078132 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:53.078132 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:53.078132 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:53.078132 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:53.078132 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:53.078132 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:53.078132 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:53.078132 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:53.078132 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:53.078132 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:53.078132 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:53.078132 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:53.078132 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:53.078132 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:53.078132 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:53.078132 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:53.079099 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:53.079099 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:53.079099 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:53.079099 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:53.079899 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 382/751] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:53.081548 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb98883-5dac-4655-98cb-ccc8930e9e9e None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:53.155260 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a6616-f203-40e8-99ac-39c8f4380d40 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:53.171330 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:53.204201 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bd39f5-30b1-42ef-aa47-89590fdd4865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:53.205209 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd39f5-30b1-42ef-aa47-89590fdd4865 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:53.205702 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd39f5-30b1-42ef-aa47-89590fdd4865 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:53.206288 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd39f5-30b1-42ef-aa47-89590fdd4865 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:53.207222 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bd39f5-30b1-42ef-aa47-89590fdd4865 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:53.207749 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bd39f5-30b1-42ef-aa47-89590fdd4865 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:53.209513 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bd39f5-30b1-42ef-aa47-89590fdd4865 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:53.228689 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 370/752] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:53.229925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bd39f5-30b1-42ef-aa47-89590fdd4865 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:53.240450 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb98883-5dac-4655-98cb-ccc8930e9e9e None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:53.276740 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e87810-18c3-490c-8373-3ee9f5d42f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:53.277574 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e87810-18c3-490c-8373-3ee9f5d42f38 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:53.277865 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e87810-18c3-490c-8373-3ee9f5d42f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:53.278147 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e87810-18c3-490c-8373-3ee9f5d42f38 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:53.278819 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e87810-18c3-490c-8373-3ee9f5d42f38 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:53.280218 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e87810-18c3-490c-8373-3ee9f5d42f38 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:53.282456 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:53.296089 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-38e87810-18c3-490c-8373-3ee9f5d42f38 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 06:06:53.296089 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:53.296089 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 06:06:53.296089 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 06:06:53.296089 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 06:06:53.296089 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 06:06:53.296089 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 06:06:53.296089 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:06:53.296089 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 06:06:53.296089 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:53.296089 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 06:06:53.296089 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:53.296089 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:53.296089 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:53.296089 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:53.296089 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:53.303949 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:53.303949 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:53.303949 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:53.303949 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:53.303949 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:53.303949 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:53.303949 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:53.303949 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 06:06:53.303949 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 06:06:53.303949 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 06:06:53.303949 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 06:06:53.303949 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:53.303949 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:53.303949 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:53.303949 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:53.303949 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:53.305698 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:53.305698 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:53.305698 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:53.305698 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:53.305698 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:53.305698 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:53.305698 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:53.305698 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:53.305698 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:53.305698 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:53.305698 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:53.305698 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 06:06:53.305698 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:06:53.305698 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 06:06:53.305698 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:53.305698 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 383/753] 10.222.0.40 () {56 vars in 1129 bytes} [Mon Jul 18 06:06:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:53.308703 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e87810-18c3-490c-8373-3ee9f5d42f38 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:53.396658 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bd39f5-30b1-42ef-aa47-89590fdd4865 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:53.417074 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e87810-18c3-490c-8373-3ee9f5d42f38 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:53.429107 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:53.447713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf985aa9-162f-4fba-ad45-7f29e794c2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:53.449324 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf985aa9-162f-4fba-ad45-7f29e794c2ab None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:53.449324 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf985aa9-162f-4fba-ad45-7f29e794c2ab None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:53.449805 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf985aa9-162f-4fba-ad45-7f29e794c2ab None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:53.451166 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf985aa9-162f-4fba-ad45-7f29e794c2ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:53.451901 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf985aa9-162f-4fba-ad45-7f29e794c2ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:53.453576 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf985aa9-162f-4fba-ad45-7f29e794c2ab None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:53.456238 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4f1ca-866a-47ab-99c1-0e4dc49d3403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:53.457172 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4f1ca-866a-47ab-99c1-0e4dc49d3403 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:53.457433 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4f1ca-866a-47ab-99c1-0e4dc49d3403 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:53.457811 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4f1ca-866a-47ab-99c1-0e4dc49d3403 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:53.467269 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda4f1ca-866a-47ab-99c1-0e4dc49d3403 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:53.469063 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda4f1ca-866a-47ab-99c1-0e4dc49d3403 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:53.471104 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 371/754] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:53.473240 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf985aa9-162f-4fba-ad45-7f29e794c2ab None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:53.486614 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-cda4f1ca-866a-47ab-99c1-0e4dc49d3403 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 06:06:53.486614 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:53.486614 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:53.486614 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:53.486614 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:53.486614 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:53.486614 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:53.486614 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:53.486614 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:53.486614 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:53.486614 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:53.486614 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:53.486614 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 06:06:53.486614 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 06:06:53.486614 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 06:06:53.486614 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 06:06:53.487834 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:53.487834 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:53.487834 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 06:06:53.487834 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:53.487834 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 06:06:53.487834 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:53.487834 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:53.487834 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:53.487834 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:53.487834 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:53.487834 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:53.487834 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:53.487834 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:53.487834 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:53.487834 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:53.487834 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:53.487834 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:53.487834 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:53.489058 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:53.489058 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:53.489058 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 06:06:53.489058 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 06:06:53.489058 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 06:06:53.489058 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 06:06:53.489058 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 06:06:53.489058 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 06:06:53.489058 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 06:06:53.489058 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:53.495804 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 384/755] 10.222.0.40 () {56 vars in 1093 bytes} [Mon Jul 18 06:06:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:53.495804 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4f1ca-866a-47ab-99c1-0e4dc49d3403 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:53.609258 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4f1ca-866a-47ab-99c1-0e4dc49d3403 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:53.616555 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:53.622688 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf985aa9-162f-4fba-ad45-7f29e794c2ab None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:53.647135 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa7d954-f5a9-4e16-b1aa-01573d8d8ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:53.648439 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7d954-f5a9-4e16-b1aa-01573d8d8ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:53.648846 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7d954-f5a9-4e16-b1aa-01573d8d8ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:53.649951 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7d954-f5a9-4e16-b1aa-01573d8d8ea1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:53.659650 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa7d954-f5a9-4e16-b1aa-01573d8d8ea1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:53.661584 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa7d954-f5a9-4e16-b1aa-01573d8d8ea1 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:53.669302 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d884de-d706-4859-bcc6-69075b15b9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:53.670522 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d884de-d706-4859-bcc6-69075b15b9d5 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:53.670957 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d884de-d706-4859-bcc6-69075b15b9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:53.671382 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d884de-d706-4859-bcc6-69075b15b9d5 None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:53.672259 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d884de-d706-4859-bcc6-69075b15b9d5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:53.673197 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d884de-d706-4859-bcc6-69075b15b9d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:53.674708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d884de-d706-4859-bcc6-69075b15b9d5 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:53.679468 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-baa7d954-f5a9-4e16-b1aa-01573d8d8ea1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 06:06:53.679468 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:53.679468 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:53.679468 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:53.679468 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:53.679468 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:53.679468 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:53.679468 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:53.679468 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:53.679468 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:53.679468 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:53.679468 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:53.679468 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 06:06:53.679468 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 06:06:53.679468 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 06:06:53.679468 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 06:06:53.680770 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:53.680770 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:53.680770 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 06:06:53.680770 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:53.680770 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 06:06:53.680770 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:53.680770 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:53.680770 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:53.680770 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:53.680770 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:53.680770 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:53.680770 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:53.680770 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:53.680770 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:53.680770 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:53.680770 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:53.680770 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:53.680770 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:53.682072 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:53.682072 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:53.682072 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 06:06:53.682072 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 06:06:53.682072 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 06:06:53.682072 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 06:06:53.682072 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 06:06:53.682072 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 06:06:53.682072 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 06:06:53.682072 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:53.683641 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 385/757] 10.222.0.40 () {56 vars in 1117 bytes} [Mon Jul 18 06:06:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:53.683843 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 372/757] 10.222.0.40 () {56 vars in 1124 bytes} [Mon Jul 18 06:06:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:53.688690 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa7d954-f5a9-4e16-b1aa-01573d8d8ea1 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:53.690012 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d884de-d706-4859-bcc6-69075b15b9d5 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:53.725726 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:53.804582 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d884de-d706-4859-bcc6-69075b15b9d5 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:53.811555 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa7d954-f5a9-4e16-b1aa-01573d8d8ea1 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:53.884564 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759bd31-84af-448d-be02-6b3407bf6143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:53.885184 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7968c4f-3b6e-4746-9522-76fad1cf3fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:53.885888 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7968c4f-3b6e-4746-9522-76fad1cf3fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:53.886180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7968c4f-3b6e-4746-9522-76fad1cf3fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:53.886464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7968c4f-3b6e-4746-9522-76fad1cf3fc2 None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:53.886677 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759bd31-84af-448d-be02-6b3407bf6143 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:53.887150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759bd31-84af-448d-be02-6b3407bf6143 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:53.887332 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7968c4f-3b6e-4746-9522-76fad1cf3fc2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:53.887636 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7968c4f-3b6e-4746-9522-76fad1cf3fc2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:53.887999 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759bd31-84af-448d-be02-6b3407bf6143 None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:53.888945 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2759bd31-84af-448d-be02-6b3407bf6143 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:53.889190 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7968c4f-3b6e-4746-9522-76fad1cf3fc2 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:53.889669 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2759bd31-84af-448d-be02-6b3407bf6143 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:53.891600 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2759bd31-84af-448d-be02-6b3407bf6143 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:53.899694 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 373/758] 10.222.0.40 () {56 vars in 1124 bytes} [Mon Jul 18 06:06:53 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:53.902191 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759bd31-84af-448d-be02-6b3407bf6143 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:53.917118 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 386/759] 10.222.0.40 () {56 vars in 1133 bytes} [Mon Jul 18 06:06:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:53.917118 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7968c4f-3b6e-4746-9522-76fad1cf3fc2 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:53.944260 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:54.027368 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759bd31-84af-448d-be02-6b3407bf6143 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:54.036113 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:54.053821 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7968c4f-3b6e-4746-9522-76fad1cf3fc2 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:54.063767 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b109e72-cd05-4943-9b7f-2552261a90cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:54.064483 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b109e72-cd05-4943-9b7f-2552261a90cc None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:54.064773 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b109e72-cd05-4943-9b7f-2552261a90cc None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:54.065053 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b109e72-cd05-4943-9b7f-2552261a90cc None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:54.065899 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b109e72-cd05-4943-9b7f-2552261a90cc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:54.066220 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b109e72-cd05-4943-9b7f-2552261a90cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:54.067847 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b109e72-cd05-4943-9b7f-2552261a90cc None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:54.088385 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 374/760] 10.222.0.40 () {56 vars in 1149 bytes} [Mon Jul 18 06:06:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:54.089269 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b109e72-cd05-4943-9b7f-2552261a90cc None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:54.106035 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0739a6a-05c7-4cce-88e7-8d6432eff199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:54.106035 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0739a6a-05c7-4cce-88e7-8d6432eff199 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:54.106035 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0739a6a-05c7-4cce-88e7-8d6432eff199 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:54.106035 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0739a6a-05c7-4cce-88e7-8d6432eff199 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:54.106879 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0739a6a-05c7-4cce-88e7-8d6432eff199 None admin] RBAC: Adding query filter params (interface=public, service_id=beb1a16838be4c68aedc7a724681d077, region_id=RegionOne) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:54.106879 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0739a6a-05c7-4cce-88e7-8d6432eff199 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:54.106879 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0739a6a-05c7-4cce-88e7-8d6432eff199 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:54.120001 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 387/761] 10.222.0.40 () {56 vars in 1251 bytes} [Mon Jul 18 06:06:53 2022] GET /identity/v3/endpoints?service_id=beb1a16838be4c68aedc7a724681d077&interface=public®ion_id=RegionOne => generated 194 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:54.120001 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f082aaec-5464-471d-8e51-a269adf8195e None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:54.120001 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f082aaec-5464-471d-8e51-a269adf8195e None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:54.120001 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f082aaec-5464-471d-8e51-a269adf8195e None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:54.122701 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 388/762] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:54.123581 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f082aaec-5464-471d-8e51-a269adf8195e None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:54.189241 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:54.206898 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b109e72-cd05-4943-9b7f-2552261a90cc None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:54.241424 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f082aaec-5464-471d-8e51-a269adf8195e None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:54.246118 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162526f-bce3-4f99-bef0-4e276367cbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:54.246985 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162526f-bce3-4f99-bef0-4e276367cbcb None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:54.247357 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162526f-bce3-4f99-bef0-4e276367cbcb None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:54.247701 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162526f-bce3-4f99-bef0-4e276367cbcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:54.259120 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8162526f-bce3-4f99-bef0-4e276367cbcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:54.260784 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8162526f-bce3-4f99-bef0-4e276367cbcb None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:54.276585 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2193216-4e26-47d9-83bd-d87d3ce78445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:54.277340 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2193216-4e26-47d9-83bd-d87d3ce78445 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:54.277754 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2193216-4e26-47d9-83bd-d87d3ce78445 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:54.278053 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2193216-4e26-47d9-83bd-d87d3ce78445 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:54.279562 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-8162526f-bce3-4f99-bef0-4e276367cbcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:54.279562 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:54.279562 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:54.279562 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:54.279562 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:54.279562 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:54.279562 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:54.279562 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:54.279562 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:54.279562 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:54.279562 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:54.279562 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:54.279562 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:54.279562 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:54.279562 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:54.280858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:54.280858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:54.280858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:54.280858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:54.280858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:54.280858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:54.280858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:54.280858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:54.280858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:54.280858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:54.280858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:54.280858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:54.280858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:54.280858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:54.280858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:54.280858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:54.280858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:54.281963 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:54.281963 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:54.281963 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:54.281963 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:54.289602 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 375/763] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:54.289602 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162526f-bce3-4f99-bef0-4e276367cbcb None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:54.312086 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:54.361929 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2193216-4e26-47d9-83bd-d87d3ce78445 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:54.364144 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2193216-4e26-47d9-83bd-d87d3ce78445 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:54.397813 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162526f-bce3-4f99-bef0-4e276367cbcb None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:54.442163 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe08c9d-68e8-41a6-b176-ba48706f6e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:54.442829 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe08c9d-68e8-41a6-b176-ba48706f6e3c None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:54.443196 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe08c9d-68e8-41a6-b176-ba48706f6e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:54.443386 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe08c9d-68e8-41a6-b176-ba48706f6e3c None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:54.444057 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe08c9d-68e8-41a6-b176-ba48706f6e3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:54.445452 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe08c9d-68e8-41a6-b176-ba48706f6e3c None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:54.448272 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:54.452357 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 376/764] 10.222.0.40 () {56 vars in 1087 bytes} [Mon Jul 18 06:06:54 2022] GET /identity/v3/services => generated 772 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:54.453478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38513f0f-a9f7-4973-8fae-d55fa1789ac4 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:54.458106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38513f0f-a9f7-4973-8fae-d55fa1789ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:54.458106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38513f0f-a9f7-4973-8fae-d55fa1789ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:54.473000 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-38513f0f-a9f7-4973-8fae-d55fa1789ac4 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:54.474309 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-38513f0f-a9f7-4973-8fae-d55fa1789ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:54.511119 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 389/765] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:06:54 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 06:06:54.513191 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2193216-4e26-47d9-83bd-d87d3ce78445 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:54.558389 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 377/766] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:06:54.559419 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38513f0f-a9f7-4973-8fae-d55fa1789ac4 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:54.590313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:54.649525 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2193216-4e26-47d9-83bd-d87d3ce78445 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:54.703909 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7160d51c-8e9a-498e-bea6-a72ed360cd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:54.705036 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160d51c-8e9a-498e-bea6-a72ed360cd53 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:54.705327 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160d51c-8e9a-498e-bea6-a72ed360cd53 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:54.705629 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160d51c-8e9a-498e-bea6-a72ed360cd53 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:54.706700 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7160d51c-8e9a-498e-bea6-a72ed360cd53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:54.707017 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7160d51c-8e9a-498e-bea6-a72ed360cd53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:54.708264 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38513f0f-a9f7-4973-8fae-d55fa1789ac4 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:54.708760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7160d51c-8e9a-498e-bea6-a72ed360cd53 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:54.722706 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 390/767] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:54.723880 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7160d51c-8e9a-498e-bea6-a72ed360cd53 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:54.760140 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:54.768372 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232a410-90d5-4cf5-b710-827110fa7234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:54.769542 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232a410-90d5-4cf5-b710-827110fa7234 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:54.769832 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232a410-90d5-4cf5-b710-827110fa7234 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:54.770125 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232a410-90d5-4cf5-b710-827110fa7234 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:54.785750 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2232a410-90d5-4cf5-b710-827110fa7234 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:54.787492 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2232a410-90d5-4cf5-b710-827110fa7234 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:54.813928 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-2232a410-90d5-4cf5-b710-827110fa7234 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:54.813928 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:54.813928 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:54.813928 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:54.813928 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:54.813928 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:54.813928 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:54.813928 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:54.813928 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:54.813928 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:54.813928 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:54.813928 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:54.813928 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:54.813928 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:54.813928 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:54.815135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:54.815135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:54.815135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:54.815135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:54.815135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:54.815135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:54.815135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:54.815135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:54.815135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:54.815135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:54.815135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:54.815135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:54.815135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:54.815135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:54.815135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:54.815135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:54.815135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:54.816370 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:54.816370 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:54.816370 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:54.816370 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:54.820489 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 378/768] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:54.822103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232a410-90d5-4cf5-b710-827110fa7234 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:54.865230 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7160d51c-8e9a-498e-bea6-a72ed360cd53 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:54.881784 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:54.918443 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039678f9-afec-43fa-9513-4f445f0b5ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:54.919485 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039678f9-afec-43fa-9513-4f445f0b5ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:54.920556 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039678f9-afec-43fa-9513-4f445f0b5ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:54.922162 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039678f9-afec-43fa-9513-4f445f0b5ae4 None admin] PATH_INFO: `/v3/services/network` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:54.922878 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039678f9-afec-43fa-9513-4f445f0b5ae4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:54.924328 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039678f9-afec-43fa-9513-4f445f0b5ae4 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:54.948404 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-039678f9-afec-43fa-9513-4f445f0b5ae4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 06:06:54.948404 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:54.948404 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 06:06:54.948404 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 06:06:54.948404 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 06:06:54.948404 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 06:06:54.948404 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 06:06:54.948404 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:06:54.948404 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 06:06:54.948404 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:54.948404 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 06:06:54.948404 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:54.948404 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:54.948404 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:54.948404 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:54.948404 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:54.948404 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:54.949511 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:54.949511 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:54.949511 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:54.949511 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:54.949511 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:54.949511 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:54.949511 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 06:06:54.949511 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 06:06:54.949511 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 06:06:54.949511 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 06:06:54.949511 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:54.949511 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:54.949511 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:54.949511 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:54.949511 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:54.949511 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:54.950371 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:54.950371 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:54.950371 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:54.950371 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:54.950371 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:54.950371 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:54.950371 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:54.950371 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:54.950371 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:54.950371 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:54.950371 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 06:06:54.950371 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:06:54.950371 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 06:06:54.950371 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:54.954395 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 391/769] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:06:54 2022] GET /identity/v3/services/network => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:54.955562 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039678f9-afec-43fa-9513-4f445f0b5ae4 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:54.974401 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232a410-90d5-4cf5-b710-827110fa7234 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:55.028464 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97db834-722c-45f8-96a3-cf1fb7c8f828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:55.029796 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97db834-722c-45f8-96a3-cf1fb7c8f828 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:55.030189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97db834-722c-45f8-96a3-cf1fb7c8f828 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:55.030575 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97db834-722c-45f8-96a3-cf1fb7c8f828 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:55.042685 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97db834-722c-45f8-96a3-cf1fb7c8f828 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:55.045593 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97db834-722c-45f8-96a3-cf1fb7c8f828 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:55.067218 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-d97db834-722c-45f8-96a3-cf1fb7c8f828 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:55.067218 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:55.067218 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:55.067218 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:55.067218 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:55.067218 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:55.067218 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:55.067218 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:55.067218 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:55.067218 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:55.067218 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:55.067218 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:55.067218 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 06:06:55.067218 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 06:06:55.067218 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 06:06:55.068358 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 06:06:55.068358 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:55.068358 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:55.068358 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:55.068358 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:55.068358 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:55.068358 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:55.068358 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:55.068358 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:55.068358 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:55.068358 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:55.068358 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:55.068358 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:55.068358 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:55.068358 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:55.068358 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:55.068358 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:55.069504 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 06:06:55.069504 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 06:06:55.069504 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 06:06:55.069504 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 06:06:55.069504 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 06:06:55.069504 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 06:06:55.069504 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:55.069504 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:55.072526 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 379/770] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:06:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:55.072526 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97db834-722c-45f8-96a3-cf1fb7c8f828 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:55.103407 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039678f9-afec-43fa-9513-4f445f0b5ae4 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:55.129435 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:55.143144 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249464b8-198e-47f4-8e2c-993a75e5232e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:55.144724 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249464b8-198e-47f4-8e2c-993a75e5232e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:55.145424 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249464b8-198e-47f4-8e2c-993a75e5232e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:55.146103 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249464b8-198e-47f4-8e2c-993a75e5232e None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:55.151025 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249464b8-198e-47f4-8e2c-993a75e5232e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:55.151025 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249464b8-198e-47f4-8e2c-993a75e5232e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:55.151025 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249464b8-198e-47f4-8e2c-993a75e5232e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:55.167452 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 392/771] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:55.168696 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249464b8-198e-47f4-8e2c-993a75e5232e None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:55.197950 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97db834-722c-45f8-96a3-cf1fb7c8f828 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:55.207910 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:55.238841 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be3595-3634-44d4-94fa-8085b9c4d5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:55.239680 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be3595-3634-44d4-94fa-8085b9c4d5c8 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:55.240070 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be3595-3634-44d4-94fa-8085b9c4d5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:55.240476 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be3595-3634-44d4-94fa-8085b9c4d5c8 None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:55.241316 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2be3595-3634-44d4-94fa-8085b9c4d5c8 None admin] RBAC: Adding query filter params (name=network) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:55.241727 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2be3595-3634-44d4-94fa-8085b9c4d5c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:55.243878 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2be3595-3634-44d4-94fa-8085b9c4d5c8 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:55.248880 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 380/772] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Jul 18 06:06:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:55.249689 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be3595-3634-44d4-94fa-8085b9c4d5c8 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:55.284951 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249464b8-198e-47f4-8e2c-993a75e5232e None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:55.323825 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05720cde-c0f0-43b9-b6dd-bdc50b12594d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:55.324458 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05720cde-c0f0-43b9-b6dd-bdc50b12594d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:55.324770 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05720cde-c0f0-43b9-b6dd-bdc50b12594d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:55.325051 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05720cde-c0f0-43b9-b6dd-bdc50b12594d None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:55.325851 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05720cde-c0f0-43b9-b6dd-bdc50b12594d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:55.326166 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05720cde-c0f0-43b9-b6dd-bdc50b12594d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:55.327605 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05720cde-c0f0-43b9-b6dd-bdc50b12594d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:55.346716 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 393/773] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 18 06:06:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:55.346716 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87999724-2ae9-431c-a570-f50f4d0e59bd None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:55.346716 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87999724-2ae9-431c-a570-f50f4d0e59bd None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:55.346716 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87999724-2ae9-431c-a570-f50f4d0e59bd None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:55.346716 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 394/774] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:55.347733 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87999724-2ae9-431c-a570-f50f4d0e59bd None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:55.370421 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be3595-3634-44d4-94fa-8085b9c4d5c8 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:55.379672 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:55.410108 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc2bc7d-7aef-480e-bb9c-8558bbb167f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:55.411032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2bc7d-7aef-480e-bb9c-8558bbb167f4 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:55.411643 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2bc7d-7aef-480e-bb9c-8558bbb167f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:55.411923 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2bc7d-7aef-480e-bb9c-8558bbb167f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:55.419896 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc2bc7d-7aef-480e-bb9c-8558bbb167f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:55.421505 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc2bc7d-7aef-480e-bb9c-8558bbb167f4 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:55.439485 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-dcc2bc7d-7aef-480e-bb9c-8558bbb167f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:55.439485 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:55.439485 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:55.439485 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:55.439485 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:55.439485 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:55.439485 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:55.439485 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:55.439485 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:55.439485 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:55.439485 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:55.439485 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:55.439485 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 06:06:55.439485 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 06:06:55.439485 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 06:06:55.440685 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 06:06:55.440685 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:55.440685 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:55.440685 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:55.440685 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:55.440685 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:55.440685 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:55.440685 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:55.440685 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:55.440685 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:55.440685 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:55.440685 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:55.440685 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:55.440685 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:55.440685 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:55.440685 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:55.440685 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:55.442409 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 06:06:55.442409 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 06:06:55.442409 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 06:06:55.442409 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 06:06:55.442409 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 06:06:55.442409 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 06:06:55.442409 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:55.442409 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:55.446938 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 381/775] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:06:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:55.447868 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc2bc7d-7aef-480e-bb9c-8558bbb167f4 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:55.465085 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:55.472922 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87999724-2ae9-431c-a570-f50f4d0e59bd None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:55.519044 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893abbb7-9a66-4963-bb83-6765299d6d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:55.519820 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893abbb7-9a66-4963-bb83-6765299d6d24 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:55.520017 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893abbb7-9a66-4963-bb83-6765299d6d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:55.520298 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893abbb7-9a66-4963-bb83-6765299d6d24 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:55.521059 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893abbb7-9a66-4963-bb83-6765299d6d24 None admin] RBAC: Adding query filter params (type=network) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:55.521489 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893abbb7-9a66-4963-bb83-6765299d6d24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:55.522982 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893abbb7-9a66-4963-bb83-6765299d6d24 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:55.533852 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 395/776] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Jul 18 06:06:55 2022] GET /identity/v3/services?type=network => generated 350 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:55.533852 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f00af-ff38-423a-9f1c-13cced86ed2c None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:55.533852 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f00af-ff38-423a-9f1c-13cced86ed2c None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:55.533852 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f00af-ff38-423a-9f1c-13cced86ed2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:55.551100 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-4c1f00af-ff38-423a-9f1c-13cced86ed2c None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:55.552291 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4c1f00af-ff38-423a-9f1c-13cced86ed2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:55.561200 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc2bc7d-7aef-480e-bb9c-8558bbb167f4 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:55.603898 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0192540-7261-4dc6-907d-b36cee075ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:55.604820 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0192540-7261-4dc6-907d-b36cee075ee1 None admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:55.605102 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0192540-7261-4dc6-907d-b36cee075ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:55.605376 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0192540-7261-4dc6-907d-b36cee075ee1 None admin] PATH_INFO: `/v3/projects/8ed50a8a3e184bde9269d7b0d9573d4a/users/f92d948810ea4241810473506b654f7a/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:55.607395 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0192540-7261-4dc6-907d-b36cee075ee1 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ed50a8a3e184bde9269d7b0d9573d4a, user_id=f92d948810ea4241810473506b654f7a, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:55.608862 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0192540-7261-4dc6-907d-b36cee075ee1 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:55.626900 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 396/777] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:06:55.627942 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f00af-ff38-423a-9f1c-13cced86ed2c None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:55.708450 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 382/778] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 18 06:06:55 2022] PUT /identity/v3/projects/8ed50a8a3e184bde9269d7b0d9573d4a/users/f92d948810ea4241810473506b654f7a/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:06:55.708450 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0192540-7261-4dc6-907d-b36cee075ee1 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:55.748708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:55.768489 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:55.780924 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f00af-ff38-423a-9f1c-13cced86ed2c None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:55.829892 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca15dd1-23a6-4aa7-b086-c958556cb13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:55.831076 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca15dd1-23a6-4aa7-b086-c958556cb13d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:55.831441 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca15dd1-23a6-4aa7-b086-c958556cb13d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:55.831825 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca15dd1-23a6-4aa7-b086-c958556cb13d None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:55.832927 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca15dd1-23a6-4aa7-b086-c958556cb13d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:55.833676 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca15dd1-23a6-4aa7-b086-c958556cb13d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:55.835324 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca15dd1-23a6-4aa7-b086-c958556cb13d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:55.856916 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0192540-7261-4dc6-907d-b36cee075ee1 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:55.857118 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 397/779] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 18 06:06:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:55.857118 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca15dd1-23a6-4aa7-b086-c958556cb13d None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:55.900872 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b9ff2-eea9-468d-bb75-0089f6a9beac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:55.901549 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b9ff2-eea9-468d-bb75-0089f6a9beac None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:55.901834 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b9ff2-eea9-468d-bb75-0089f6a9beac None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:55.902108 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b9ff2-eea9-468d-bb75-0089f6a9beac None admin] PATH_INFO: `/v3/endpoints` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:55.902938 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68b9ff2-eea9-468d-bb75-0089f6a9beac None admin] RBAC: Adding query filter params (interface=public, service_id=546a11353a904cfd8105103f042d7883, region_id=RegionOne) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:55.903249 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68b9ff2-eea9-468d-bb75-0089f6a9beac None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:55.905420 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68b9ff2-eea9-468d-bb75-0089f6a9beac None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:55.916746 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 383/780] 10.222.0.40 () {56 vars in 1251 bytes} [Mon Jul 18 06:06:55 2022] GET /identity/v3/endpoints?service_id=546a11353a904cfd8105103f042d7883&interface=public®ion_id=RegionOne => generated 194 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:55.917757 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b9ff2-eea9-468d-bb75-0089f6a9beac None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:55.986500 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca15dd1-23a6-4aa7-b086-c958556cb13d None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:56.030899 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a2dfe-d373-4c17-89ae-1742caf874f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:56.031620 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:56.031770 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a2dfe-d373-4c17-89ae-1742caf874f4 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:56.031998 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a2dfe-d373-4c17-89ae-1742caf874f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:56.032285 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a2dfe-d373-4c17-89ae-1742caf874f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:56.042966 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b9ff2-eea9-468d-bb75-0089f6a9beac None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:56.050768 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0a2dfe-d373-4c17-89ae-1742caf874f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:56.059492 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-cf0a2dfe-d373-4c17-89ae-1742caf874f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:06:56.059492 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:56.059492 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:56.059492 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:56.059492 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:56.059492 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:56.059492 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:56.059492 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:56.059492 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:56.059492 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:56.059492 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:56.059492 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:56.059492 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:56.059492 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:56.059492 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:56.059492 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:56.060648 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:56.060648 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:56.060648 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:56.060648 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:56.060648 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:56.060648 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:56.060648 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:56.060648 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:56.060648 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:56.060648 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:56.060648 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:56.060648 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:56.060648 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:56.060648 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:56.060648 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:56.060648 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:56.061790 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:56.061790 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:56.061790 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:56.061790 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:56.061790 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:56.061790 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:56.061790 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:56.061790 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:56.061790 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:56.061790 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:56.061790 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:06:56.061790 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:56.061790 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 398/781] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 18 06:06:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:56.062627 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a2dfe-d373-4c17-89ae-1742caf874f4 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:56.081875 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:56.090803 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067400a6-4792-4522-b0fd-48b8d31b5e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:56.091616 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067400a6-4792-4522-b0fd-48b8d31b5e37 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:56.092001 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067400a6-4792-4522-b0fd-48b8d31b5e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:56.092441 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067400a6-4792-4522-b0fd-48b8d31b5e37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:56.093532 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067400a6-4792-4522-b0fd-48b8d31b5e37 None admin] RBAC: Adding query filter params (role.id=ab10744ff77646eab23a769ab89c74aa, scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a, user.id=b84dde57ff8e47419866377a819bea21) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:56.093988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067400a6-4792-4522-b0fd-48b8d31b5e37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:56.098376 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067400a6-4792-4522-b0fd-48b8d31b5e37 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:56.108651 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 384/782] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:06:55 2022] GET /identity/v3/role_assignments?user.id=b84dde57ff8e47419866377a819bea21&scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a&role.id=ab10744ff77646eab23a769ab89c74aa => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:56.109773 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067400a6-4792-4522-b0fd-48b8d31b5e37 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:56.210617 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a2dfe-d373-4c17-89ae-1742caf874f4 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:56.279233 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067400a6-4792-4522-b0fd-48b8d31b5e37 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:56.285313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514350f-e63f-405a-a66a-850ddbcff6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:56.286020 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514350f-e63f-405a-a66a-850ddbcff6aa None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:56.286541 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514350f-e63f-405a-a66a-850ddbcff6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:56.286828 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514350f-e63f-405a-a66a-850ddbcff6aa None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:56.287539 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e514350f-e63f-405a-a66a-850ddbcff6aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:56.289026 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e514350f-e63f-405a-a66a-850ddbcff6aa None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:56.300870 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 399/783] 10.222.0.40 () {56 vars in 1087 bytes} [Mon Jul 18 06:06:56 2022] GET /identity/v3/services => generated 772 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:56.334479 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31204958-c296-4abd-a744-15469f53d807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:56.335412 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31204958-c296-4abd-a744-15469f53d807 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:56.335882 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31204958-c296-4abd-a744-15469f53d807 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:56.336159 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31204958-c296-4abd-a744-15469f53d807 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:56.337939 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31204958-c296-4abd-a744-15469f53d807 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:56.338319 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31204958-c296-4abd-a744-15469f53d807 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:56.339730 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31204958-c296-4abd-a744-15469f53d807 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:56.350373 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 385/784] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:06:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:56.356649 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514350f-e63f-405a-a66a-850ddbcff6aa None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:56.452322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:56.482050 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514350f-e63f-405a-a66a-850ddbcff6aa None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:56.522261 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b677d-9db2-467e-977f-2eaa9470c69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:56.523088 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b677d-9db2-467e-977f-2eaa9470c69c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:56.523373 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b677d-9db2-467e-977f-2eaa9470c69c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:56.523654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b677d-9db2-467e-977f-2eaa9470c69c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:56.537345 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b677d-9db2-467e-977f-2eaa9470c69c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:56.538886 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b677d-9db2-467e-977f-2eaa9470c69c None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:56.559119 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-956b677d-9db2-467e-977f-2eaa9470c69c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:56.559119 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:56.559119 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:56.559119 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:56.559119 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:56.559119 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:56.559119 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:56.559119 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:56.559119 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:56.559119 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:56.559119 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:56.559119 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:56.559119 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:56.559119 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:56.559119 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:56.560104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:56.560104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:56.560104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:56.560104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:56.560104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:56.560104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:56.560104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:56.560104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:56.560104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:56.560104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:56.560104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:56.560104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:56.560104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:56.560104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:56.560104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:56.560104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:56.560104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:56.561027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:56.561027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:56.561027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:56.561027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:56.561027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 400/785] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:06:56.569582 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31204958-c296-4abd-a744-15469f53d807 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:56.597111 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:56.698803 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31204958-c296-4abd-a744-15469f53d807 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:56.723370 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e83560-a58e-46af-b2e7-1c4064f0f215 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:56.723704 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e83560-a58e-46af-b2e7-1c4064f0f215 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:56.723841 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e83560-a58e-46af-b2e7-1c4064f0f215 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:56.726483 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 401/786] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:06:56.733042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e344b3-5b4d-4257-8248-459f930926d9 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:56.733323 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e344b3-5b4d-4257-8248-459f930926d9 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:56.733604 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e344b3-5b4d-4257-8248-459f930926d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:56.741457 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969be039-d2ab-49b1-8bce-4d7c0a3318a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:56.742132 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969be039-d2ab-49b1-8bce-4d7c0a3318a7 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:56.742417 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969be039-d2ab-49b1-8bce-4d7c0a3318a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:56.743230 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969be039-d2ab-49b1-8bce-4d7c0a3318a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:56.744024 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969be039-d2ab-49b1-8bce-4d7c0a3318a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:56.744328 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969be039-d2ab-49b1-8bce-4d7c0a3318a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:56.745723 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969be039-d2ab-49b1-8bce-4d7c0a3318a7 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:56.755470 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-74e344b3-5b4d-4257-8248-459f930926d9 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:56.756713 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-74e344b3-5b4d-4257-8248-459f930926d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:56.759782 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 386/787] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:56.767764 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969be039-d2ab-49b1-8bce-4d7c0a3318a7 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:56.851865 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:56.876942 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969be039-d2ab-49b1-8bce-4d7c0a3318a7 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:56.912538 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 402/788] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:06:56.926156 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13332795-3294-41eb-81ee-9bb70afe9d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:56.927343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13332795-3294-41eb-81ee-9bb70afe9d25 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:56.927838 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13332795-3294-41eb-81ee-9bb70afe9d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:56.928344 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13332795-3294-41eb-81ee-9bb70afe9d25 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:56.931267 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e344b3-5b4d-4257-8248-459f930926d9 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:56.949634 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13332795-3294-41eb-81ee-9bb70afe9d25 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:56.951481 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13332795-3294-41eb-81ee-9bb70afe9d25 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:56.971708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-13332795-3294-41eb-81ee-9bb70afe9d25 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 06:06:56.971708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:56.971708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:56.971708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:56.971708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:56.971708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:56.971708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:56.971708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:56.971708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:56.971708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:56.971708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:56.971708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:56.971708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 06:06:56.971708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 06:06:56.971708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 06:06:56.971708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 06:06:56.972801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:56.972801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:56.972801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 06:06:56.972801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:56.972801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 06:06:56.972801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:56.972801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:56.972801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:56.972801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:56.972801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:56.972801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:56.972801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:56.972801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:56.972801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:56.972801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:56.972801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:56.972801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:56.972801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:56.973801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:56.973801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:56.973801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 06:06:56.973801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 06:06:56.973801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 06:06:56.973801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 06:06:56.973801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 06:06:56.973801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 06:06:56.973801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 06:06:56.973801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:56.981064 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 387/789] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 18 06:06:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:56.985434 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13332795-3294-41eb-81ee-9bb70afe9d25 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:57.010854 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:57.094454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e344b3-5b4d-4257-8248-459f930926d9 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:57.159147 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90b10f-95bd-43f1-bbe5-b5580420d590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:57.159988 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90b10f-95bd-43f1-bbe5-b5580420d590 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:57.160264 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90b10f-95bd-43f1-bbe5-b5580420d590 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:57.160566 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90b10f-95bd-43f1-bbe5-b5580420d590 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:57.161494 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e90b10f-95bd-43f1-bbe5-b5580420d590 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:57.162892 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e90b10f-95bd-43f1-bbe5-b5580420d590 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:57.183211 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-6e90b10f-95bd-43f1-bbe5-b5580420d590 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 06:06:57.183211 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:57.183211 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 06:06:57.183211 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 06:06:57.183211 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 06:06:57.183211 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 06:06:57.183211 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 06:06:57.183211 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:06:57.183211 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 06:06:57.183211 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:57.183211 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 06:06:57.183211 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:57.183211 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:57.183211 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:57.183211 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:57.183211 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:57.184235 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:57.184235 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:57.184235 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:57.184235 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:57.184235 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:57.184235 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:57.184235 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:57.184235 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 06:06:57.184235 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 06:06:57.184235 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 06:06:57.184235 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 06:06:57.184235 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:57.184235 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:57.184235 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:57.184235 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:57.184235 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:57.185141 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:57.185141 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:57.185141 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:57.185141 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:57.185141 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:57.185141 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:57.185141 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:57.185141 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:57.185141 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:57.185141 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:57.185141 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:57.185141 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 06:06:57.185141 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:06:57.185141 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 06:06:57.185141 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:57.189728 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13332795-3294-41eb-81ee-9bb70afe9d25 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:57.192940 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 403/790] 10.222.0.40 () {56 vars in 1129 bytes} [Mon Jul 18 06:06:56 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:06:57.199923 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90b10f-95bd-43f1-bbe5-b5580420d590 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:57.228315 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c1f96-0fa7-4181-a4c3-0035ff5a9e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:57.229491 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c1f96-0fa7-4181-a4c3-0035ff5a9e47 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:57.230009 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c1f96-0fa7-4181-a4c3-0035ff5a9e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:57.230490 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c1f96-0fa7-4181-a4c3-0035ff5a9e47 None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:57.231541 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c1f96-0fa7-4181-a4c3-0035ff5a9e47 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:57.232089 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c1f96-0fa7-4181-a4c3-0035ff5a9e47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:57.233823 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c1f96-0fa7-4181-a4c3-0035ff5a9e47 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:57.257031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 388/791] 10.222.0.40 () {56 vars in 1135 bytes} [Mon Jul 18 06:06:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:57.257351 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:57.265482 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c1f96-0fa7-4181-a4c3-0035ff5a9e47 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:57.391085 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90b10f-95bd-43f1-bbe5-b5580420d590 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:57.431526 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:57.463801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c185747-1381-46c1-a463-e6dc7bbdda37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:57.465506 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c185747-1381-46c1-a463-e6dc7bbdda37 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:57.466075 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c185747-1381-46c1-a463-e6dc7bbdda37 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:57.466687 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c185747-1381-46c1-a463-e6dc7bbdda37 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:57.469569 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c185747-1381-46c1-a463-e6dc7bbdda37 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:57.470670 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c185747-1381-46c1-a463-e6dc7bbdda37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:57.472598 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c185747-1381-46c1-a463-e6dc7bbdda37 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:57.483008 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 404/792] 10.222.0.40 () {56 vars in 1124 bytes} [Mon Jul 18 06:06:57 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:06:57.494337 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c185747-1381-46c1-a463-e6dc7bbdda37 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:57.559730 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c1f96-0fa7-4181-a4c3-0035ff5a9e47 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:57.673983 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c185747-1381-46c1-a463-e6dc7bbdda37 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:57.701235 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba13d0-9ff9-465e-a7da-9786b88c6966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:57.702073 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba13d0-9ff9-465e-a7da-9786b88c6966 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:57.702363 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba13d0-9ff9-465e-a7da-9786b88c6966 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:57.702633 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba13d0-9ff9-465e-a7da-9786b88c6966 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:57.720531 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9441772-32d6-4a18-9501-2bc01831ecee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:57.721225 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9441772-32d6-4a18-9501-2bc01831ecee None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:57.721509 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9441772-32d6-4a18-9501-2bc01831ecee None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:57.721788 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9441772-32d6-4a18-9501-2bc01831ecee None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:57.722606 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9441772-32d6-4a18-9501-2bc01831ecee None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:57.722921 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9441772-32d6-4a18-9501-2bc01831ecee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:57.724322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9441772-32d6-4a18-9501-2bc01831ecee None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:57.737800 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 405/793] 10.222.0.40 () {56 vars in 1124 bytes} [Mon Jul 18 06:06:57 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:57.739203 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9441772-32d6-4a18-9501-2bc01831ecee None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:57.753850 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbba13d0-9ff9-465e-a7da-9786b88c6966 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:57.756638 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbba13d0-9ff9-465e-a7da-9786b88c6966 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:57.804478 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:57.824955 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-cbba13d0-9ff9-465e-a7da-9786b88c6966 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:57.824955 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:57.824955 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:57.824955 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:57.824955 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:57.824955 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:57.824955 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:57.824955 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:57.824955 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:57.824955 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:57.824955 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:57.824955 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:57.824955 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:57.824955 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:57.824955 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:57.826478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:57.826478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:57.826478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:57.826478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:57.826478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:57.826478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:57.826478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:57.826478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:57.826478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:57.826478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:57.826478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:57.826478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:57.826478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:57.826478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:57.826478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:57.826478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:57.826478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:57.828647 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:57.828647 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:57.828647 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:57.828647 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:57.833007 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 389/794] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 568 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:57.839380 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba13d0-9ff9-465e-a7da-9786b88c6966 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:57.874272 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9441772-32d6-4a18-9501-2bc01831ecee None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:57.921134 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30b01e6-1611-49d6-a99d-334527f20186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:57.921812 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b01e6-1611-49d6-a99d-334527f20186 None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:57.922095 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b01e6-1611-49d6-a99d-334527f20186 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:57.922372 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b01e6-1611-49d6-a99d-334527f20186 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:57.923133 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30b01e6-1611-49d6-a99d-334527f20186 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:57.928277 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30b01e6-1611-49d6-a99d-334527f20186 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:57.973766 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:58.037612 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba13d0-9ff9-465e-a7da-9786b88c6966 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:58.064495 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 406/795] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Jul 18 06:06:57 2022] POST /identity/v3/endpoints => generated 346 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:06:58.101290 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163d48f-c296-4733-93f9-f3f1a4da81c6 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:58.101645 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163d48f-c296-4733-93f9-f3f1a4da81c6 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:58.101856 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163d48f-c296-4733-93f9-f3f1a4da81c6 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:58.106127 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 407/796] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:58.109265 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a46fc8-08a7-48cc-aadf-3428d45e2313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:58.109769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a46fc8-08a7-48cc-aadf-3428d45e2313 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:58.110055 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a46fc8-08a7-48cc-aadf-3428d45e2313 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:58.110333 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a46fc8-08a7-48cc-aadf-3428d45e2313 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:58.111472 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a46fc8-08a7-48cc-aadf-3428d45e2313 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:58.111472 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a46fc8-08a7-48cc-aadf-3428d45e2313 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:58.111832 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97a0fb-4798-4335-ac0a-e445ff6402fc None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:58.111986 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97a0fb-4798-4335-ac0a-e445ff6402fc None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:58.112147 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97a0fb-4798-4335-ac0a-e445ff6402fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:58.112754 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a46fc8-08a7-48cc-aadf-3428d45e2313 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:58.128953 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 390/797] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:06:58.136400 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a46fc8-08a7-48cc-aadf-3428d45e2313 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:58.157611 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-ce97a0fb-4798-4335-ac0a-e445ff6402fc None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:58.159108 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ce97a0fb-4798-4335-ac0a-e445ff6402fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:58.272927 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a46fc8-08a7-48cc-aadf-3428d45e2313 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:58.279098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:58.350570 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 408/798] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:06:58.354325 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af8abb-a3a6-4714-8416-fb05f25f4820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:58.355522 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af8abb-a3a6-4714-8416-fb05f25f4820 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:58.356045 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af8abb-a3a6-4714-8416-fb05f25f4820 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:58.356568 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af8abb-a3a6-4714-8416-fb05f25f4820 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:58.365323 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43af8abb-a3a6-4714-8416-fb05f25f4820 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:58.367162 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43af8abb-a3a6-4714-8416-fb05f25f4820 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:58.377195 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce97a0fb-4798-4335-ac0a-e445ff6402fc None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:58.384725 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-43af8abb-a3a6-4714-8416-fb05f25f4820 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:58.384725 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:58.384725 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:58.384725 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:58.384725 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:58.384725 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:58.384725 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:58.384725 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:58.384725 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:58.384725 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:58.384725 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:58.384725 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:58.384725 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 06:06:58.384725 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 06:06:58.384725 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 06:06:58.385707 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 06:06:58.385707 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:58.385707 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:58.385707 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:58.385707 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:58.385707 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:58.385707 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:58.385707 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:58.385707 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:58.385707 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:58.385707 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:58.385707 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:58.385707 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:58.385707 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:58.385707 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:58.385707 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:58.385707 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:58.386597 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 06:06:58.386597 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 06:06:58.386597 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 06:06:58.386597 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 06:06:58.386597 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 06:06:58.386597 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 06:06:58.386597 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:06:58.386597 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:58.386597 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 391/799] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:06:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:06:58.393583 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af8abb-a3a6-4714-8416-fb05f25f4820 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:58.429256 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:58.527439 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce97a0fb-4798-4335-ac0a-e445ff6402fc None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:58.545415 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af8abb-a3a6-4714-8416-fb05f25f4820 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:58.576591 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb80b59-ea9f-4846-b109-ec1d96ac39e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:58.577411 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb80b59-ea9f-4846-b109-ec1d96ac39e9 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:58.577701 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb80b59-ea9f-4846-b109-ec1d96ac39e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:58.578141 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb80b59-ea9f-4846-b109-ec1d96ac39e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:58.591105 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63872334-7210-4b2b-9a9e-be4369641d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:58.592600 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63872334-7210-4b2b-9a9e-be4369641d3a None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:58.593101 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63872334-7210-4b2b-9a9e-be4369641d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:58.593621 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63872334-7210-4b2b-9a9e-be4369641d3a None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:58.594685 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb80b59-ea9f-4846-b109-ec1d96ac39e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:06:58.595145 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63872334-7210-4b2b-9a9e-be4369641d3a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:58.595663 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63872334-7210-4b2b-9a9e-be4369641d3a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:58.597354 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63872334-7210-4b2b-9a9e-be4369641d3a None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:58.604350 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-bcb80b59-ea9f-4846-b109-ec1d96ac39e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:06:58.604350 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:58.604350 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:58.604350 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:58.604350 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:58.604350 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:58.604350 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:58.604350 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:58.604350 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:58.604350 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:58.604350 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:58.604350 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:58.604350 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:06:58.604350 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:06:58.604350 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:06:58.604350 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:06:58.605483 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:06:58.605483 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:06:58.605483 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:06:58.605483 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:06:58.605483 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:58.605483 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:58.605483 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:58.605483 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:58.605483 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:58.605483 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:58.605483 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:58.605483 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:58.605483 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:58.605483 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:58.605483 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:58.605483 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:58.606334 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:58.606334 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:58.606334 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:58.606334 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:58.606334 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:06:58.606334 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:06:58.606334 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:06:58.606334 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:06:58.606334 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:06:58.606334 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:06:58.606334 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:06:58.606334 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:58.606334 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 409/800] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 18 06:06:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:06:58.616471 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb80b59-ea9f-4846-b109-ec1d96ac39e9 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:58.626124 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 392/801] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 18 06:06:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:06:58.635638 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63872334-7210-4b2b-9a9e-be4369641d3a None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:58.663950 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:58.770240 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63872334-7210-4b2b-9a9e-be4369641d3a None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:58.846331 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fea01e-88f5-473e-806d-aa12c741428b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:58.847591 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fea01e-88f5-473e-806d-aa12c741428b None admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:58.848099 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fea01e-88f5-473e-806d-aa12c741428b None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:58.848592 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fea01e-88f5-473e-806d-aa12c741428b None admin] PATH_INFO: `/v3/projects/8ed50a8a3e184bde9269d7b0d9573d4a/users/0681f5fbea6d415c9ad2f3afae256a5c/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:58.855111 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fea01e-88f5-473e-806d-aa12c741428b None admin] RBAC: Authorizing `identity:create_grant(project_id=8ed50a8a3e184bde9269d7b0d9573d4a, user_id=0681f5fbea6d415c9ad2f3afae256a5c, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:58.856954 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fea01e-88f5-473e-806d-aa12c741428b None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:58.895965 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:58.910775 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb80b59-ea9f-4846-b109-ec1d96ac39e9 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:58.938467 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 393/802] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 18 06:06:58 2022] PUT /identity/v3/projects/8ed50a8a3e184bde9269d7b0d9573d4a/users/0681f5fbea6d415c9ad2f3afae256a5c/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:06:58.982479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d998218-354a-4f72-875e-306d8262cd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:58.983658 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d998218-354a-4f72-875e-306d8262cd43 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:58.984537 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d998218-354a-4f72-875e-306d8262cd43 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:58.985048 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d998218-354a-4f72-875e-306d8262cd43 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:58.986923 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d998218-354a-4f72-875e-306d8262cd43 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:58.987475 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d998218-354a-4f72-875e-306d8262cd43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:58.990130 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d998218-354a-4f72-875e-306d8262cd43 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:59.007853 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 410/803] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:06:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:06:59.013646 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fea01e-88f5-473e-806d-aa12c741428b None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:59.163134 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:59.188834 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fea01e-88f5-473e-806d-aa12c741428b None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:59.253326 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc61aa4-7b5a-451c-8994-f7c6ccbf1956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:59.254164 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc61aa4-7b5a-451c-8994-f7c6ccbf1956 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:59.254446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc61aa4-7b5a-451c-8994-f7c6ccbf1956 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:59.254732 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc61aa4-7b5a-451c-8994-f7c6ccbf1956 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:59.257752 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10512fc6-9cec-4594-9ba5-282d98d358f4 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:59.258058 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10512fc6-9cec-4594-9ba5-282d98d358f4 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:59.258397 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10512fc6-9cec-4594-9ba5-282d98d358f4 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:59.260275 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 411/804] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:59.268989 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c079221-efde-404c-a338-1efe18865303 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:59.269323 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c079221-efde-404c-a338-1efe18865303 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:59.269543 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c079221-efde-404c-a338-1efe18865303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:59.272157 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc61aa4-7b5a-451c-8994-f7c6ccbf1956 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:59.273690 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc61aa4-7b5a-451c-8994-f7c6ccbf1956 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:59.306398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-4dc61aa4-7b5a-451c-8994-f7c6ccbf1956 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:59.306398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:59.306398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:59.306398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:59.306398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:59.306398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:59.306398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:59.306398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:59.306398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:59.306398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:59.306398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:59.306398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:59.306398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:06:59.306398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:06:59.306398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:06:59.307774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:06:59.307774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:59.307774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:59.307774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:59.307774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:59.307774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:59.307774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:59.307774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:59.307774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:59.307774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:59.307774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:59.307774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:59.307774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:59.307774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:59.307774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:59.307774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:59.307774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:59.308922 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:06:59.308922 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:06:59.308922 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:06:59.308922 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:59.308922 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 394/805] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:06:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:59.309219 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-8c079221-efde-404c-a338-1efe18865303 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:59.310894 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-8c079221-efde-404c-a338-1efe18865303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:59.313835 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc61aa4-7b5a-451c-8994-f7c6ccbf1956 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:59.428623 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 412/806] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:06:59.436403 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a647c-a631-40a6-b9f4-4fba6f70457f None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:59.436403 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a647c-a631-40a6-b9f4-4fba6f70457f None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:59.436403 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a647c-a631-40a6-b9f4-4fba6f70457f None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:59.436403 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 413/807] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:06:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:06:59.455282 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d2a48-df16-40f9-a8a9-51501ce030a5 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:59.455589 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d2a48-df16-40f9-a8a9-51501ce030a5 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:59.455857 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d2a48-df16-40f9-a8a9-51501ce030a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:59.505936 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-d52d2a48-df16-40f9-a8a9-51501ce030a5 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:06:59.507151 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d52d2a48-df16-40f9-a8a9-51501ce030a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:06:59.537084 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc61aa4-7b5a-451c-8994-f7c6ccbf1956 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:59.579476 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956bfc98-6d54-4f3f-a18b-54dcc36fcf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:59.580629 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956bfc98-6d54-4f3f-a18b-54dcc36fcf8e None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:59.581180 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956bfc98-6d54-4f3f-a18b-54dcc36fcf8e None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:59.581864 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956bfc98-6d54-4f3f-a18b-54dcc36fcf8e None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:59.583233 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956bfc98-6d54-4f3f-a18b-54dcc36fcf8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:06:59.583353 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956bfc98-6d54-4f3f-a18b-54dcc36fcf8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:59.585488 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956bfc98-6d54-4f3f-a18b-54dcc36fcf8e None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:59.589061 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:59.600504 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 395/808] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:06:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:06:59.601569 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956bfc98-6d54-4f3f-a18b-54dcc36fcf8e None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:59.671052 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 414/809] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:06:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:06:59.673807 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d2a48-df16-40f9-a8a9-51501ce030a5 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:59.730609 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:06:59.847456 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956bfc98-6d54-4f3f-a18b-54dcc36fcf8e None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:59.884373 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d2a48-df16-40f9-a8a9-51501ce030a5 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:06:59.890288 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5193af-8984-486e-aa97-048af3443f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:59.891671 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5193af-8984-486e-aa97-048af3443f4c None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:59.892177 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5193af-8984-486e-aa97-048af3443f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:59.892619 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5193af-8984-486e-aa97-048af3443f4c None admin] PATH_INFO: `/v3/services/network` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:59.893742 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5193af-8984-486e-aa97-048af3443f4c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:59.895262 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5193af-8984-486e-aa97-048af3443f4c None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:59.911412 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-0b5193af-8984-486e-aa97-048af3443f4c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 06:06:59.911412 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:59.911412 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 06:06:59.911412 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 06:06:59.911412 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 06:06:59.911412 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 06:06:59.911412 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 06:06:59.911412 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:06:59.911412 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 06:06:59.911412 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:59.911412 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 06:06:59.911412 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:59.911412 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:59.911412 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:59.911412 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:59.911412 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:59.911412 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:59.912532 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:59.912532 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:59.912532 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:59.912532 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:59.912532 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:59.912532 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:59.912532 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 06:06:59.912532 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 06:06:59.912532 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 06:06:59.912532 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 06:06:59.912532 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:59.912532 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:59.912532 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:59.912532 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:59.912532 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:59.912532 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:59.913393 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:59.913393 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:59.913393 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:59.913393 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:59.913393 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:59.913393 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:59.913393 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:59.913393 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:59.913393 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:59.913393 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:59.913393 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 06:06:59.913393 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:06:59.913393 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 06:06:59.913393 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:06:59.916834 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 396/810] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:06:59 2022] GET /identity/v3/services/network => generated 88 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:59.918105 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5193af-8984-486e-aa97-048af3443f4c None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:59.930117 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e23ffb-50f7-4bd3-b112-ca8af2a1b816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:06:59.931291 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e23ffb-50f7-4bd3-b112-ca8af2a1b816 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:06:59.931757 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e23ffb-50f7-4bd3-b112-ca8af2a1b816 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:06:59.932255 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e23ffb-50f7-4bd3-b112-ca8af2a1b816 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:06:59.941656 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e23ffb-50f7-4bd3-b112-ca8af2a1b816 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:06:59.943667 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e23ffb-50f7-4bd3-b112-ca8af2a1b816 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:06:59.960641 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-39e23ffb-50f7-4bd3-b112-ca8af2a1b816 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 06:06:59.960641 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:06:59.960641 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:06:59.960641 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:06:59.960641 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:06:59.960641 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:06:59.960641 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:06:59.960641 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:06:59.960641 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:06:59.960641 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:06:59.960641 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:06:59.960641 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:06:59.960641 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 06:06:59.960641 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 06:06:59.960641 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 06:06:59.960641 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 06:06:59.962402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:06:59.962402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:06:59.962402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 06:06:59.962402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:59.962402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 06:06:59.962402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:06:59.962402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:06:59.962402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:06:59.962402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:06:59.962402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:06:59.962402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:06:59.962402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:06:59.962402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:06:59.962402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:06:59.962402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:06:59.962402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:06:59.962402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:06:59.962402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:06:59.963551 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:06:59.963551 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:06:59.963551 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 06:06:59.963551 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 06:06:59.963551 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 06:06:59.963551 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 06:06:59.963551 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 06:06:59.963551 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 06:06:59.963551 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 06:06:59.963551 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:06:59.966163 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 415/811] 10.222.0.40 () {56 vars in 1093 bytes} [Mon Jul 18 06:06:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:06:59.967274 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e23ffb-50f7-4bd3-b112-ca8af2a1b816 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:06:59.998574 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:00.045750 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5193af-8984-486e-aa97-048af3443f4c None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:00.062634 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:00.087945 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867173c-6c57-42fc-9a29-d607eda0ec44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:00.089115 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867173c-6c57-42fc-9a29-d607eda0ec44 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:00.089589 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867173c-6c57-42fc-9a29-d607eda0ec44 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:00.089872 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867173c-6c57-42fc-9a29-d607eda0ec44 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:00.094152 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e23ffb-50f7-4bd3-b112-ca8af2a1b816 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:00.102745 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f867173c-6c57-42fc-9a29-d607eda0ec44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:07:00.111164 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-f867173c-6c57-42fc-9a29-d607eda0ec44 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:07:00.111164 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:00.111164 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:00.111164 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:00.111164 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:00.111164 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:00.111164 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:00.111164 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:00.111164 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:00.111164 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:00.111164 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:00.111164 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:00.111164 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:07:00.111164 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:07:00.111164 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:07:00.111164 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:07:00.112362 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:07:00.112362 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:07:00.112362 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:07:00.112362 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:07:00.112362 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:00.112362 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:00.112362 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:00.112362 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:00.112362 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:00.112362 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:00.112362 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:00.112362 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:00.112362 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:00.112362 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:00.112362 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:00.112362 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:00.113456 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:00.113456 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:00.113456 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:00.113456 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:00.113456 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:07:00.113456 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:07:00.113456 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:07:00.113456 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:07:00.113456 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:07:00.113456 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:07:00.113456 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:07:00.113456 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:00.115502 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 397/812] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 18 06:06:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:00.116519 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867173c-6c57-42fc-9a29-d607eda0ec44 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:00.142001 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf578b9-645a-44a4-b36a-c42bdd009c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:00.143049 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf578b9-645a-44a4-b36a-c42bdd009c50 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:00.143503 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf578b9-645a-44a4-b36a-c42bdd009c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:00.143978 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf578b9-645a-44a4-b36a-c42bdd009c50 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:00.144975 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf578b9-645a-44a4-b36a-c42bdd009c50 None admin] RBAC: Adding query filter params (name=network) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:00.145455 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf578b9-645a-44a4-b36a-c42bdd009c50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:00.147366 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf578b9-645a-44a4-b36a-c42bdd009c50 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:00.153936 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 416/813] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Jul 18 06:06:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:00.155082 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf578b9-645a-44a4-b36a-c42bdd009c50 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:00.236104 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867173c-6c57-42fc-9a29-d607eda0ec44 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:00.255704 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:00.276511 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe0d1f-ba3e-49c8-a33f-9f856917cf8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:00.277373 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe0d1f-ba3e-49c8-a33f-9f856917cf8f None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:00.277761 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe0d1f-ba3e-49c8-a33f-9f856917cf8f None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:00.277955 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf578b9-645a-44a4-b36a-c42bdd009c50 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:00.278229 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe0d1f-ba3e-49c8-a33f-9f856917cf8f None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:00.279207 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fe0d1f-ba3e-49c8-a33f-9f856917cf8f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:00.279669 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fe0d1f-ba3e-49c8-a33f-9f856917cf8f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:00.282184 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fe0d1f-ba3e-49c8-a33f-9f856917cf8f None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:00.302667 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 398/814] 10.222.0.40 () {56 vars in 1133 bytes} [Mon Jul 18 06:07:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:00.303865 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe0d1f-ba3e-49c8-a33f-9f856917cf8f None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:00.326350 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508307c7-27a0-4135-9afc-64a0d535adb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:00.327630 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508307c7-27a0-4135-9afc-64a0d535adb9 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:00.328091 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508307c7-27a0-4135-9afc-64a0d535adb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:00.328559 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508307c7-27a0-4135-9afc-64a0d535adb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:00.329497 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508307c7-27a0-4135-9afc-64a0d535adb9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:00.330135 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508307c7-27a0-4135-9afc-64a0d535adb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:00.331893 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508307c7-27a0-4135-9afc-64a0d535adb9 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:00.344606 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 417/815] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 18 06:07:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:00.345839 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508307c7-27a0-4135-9afc-64a0d535adb9 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:00.349601 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:00.425793 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe0d1f-ba3e-49c8-a33f-9f856917cf8f None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:00.469564 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76910a42-2a93-4e8d-8a43-dde6cb64b8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:00.470369 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76910a42-2a93-4e8d-8a43-dde6cb64b8b2 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:00.470721 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76910a42-2a93-4e8d-8a43-dde6cb64b8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:00.471069 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76910a42-2a93-4e8d-8a43-dde6cb64b8b2 None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:00.471879 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76910a42-2a93-4e8d-8a43-dde6cb64b8b2 None admin] RBAC: Adding query filter params (type=network) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:00.472275 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76910a42-2a93-4e8d-8a43-dde6cb64b8b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:00.474098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76910a42-2a93-4e8d-8a43-dde6cb64b8b2 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:00.485615 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 399/816] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Jul 18 06:07:00 2022] GET /identity/v3/services?type=network => generated 350 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:00.486856 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76910a42-2a93-4e8d-8a43-dde6cb64b8b2 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:00.541461 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:00.598681 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508307c7-27a0-4135-9afc-64a0d535adb9 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:00.615083 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76910a42-2a93-4e8d-8a43-dde6cb64b8b2 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:00.640495 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e892f-b297-4742-8e0d-cee9f39d4eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:00.641504 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e892f-b297-4742-8e0d-cee9f39d4eb4 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:00.641978 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e892f-b297-4742-8e0d-cee9f39d4eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:00.642442 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e892f-b297-4742-8e0d-cee9f39d4eb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:00.659404 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d07e6ad-d1da-4229-8d0b-8bbc5813c3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:00.660497 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07e6ad-d1da-4229-8d0b-8bbc5813c3a6 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:00.660784 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07e6ad-d1da-4229-8d0b-8bbc5813c3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:00.661059 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07e6ad-d1da-4229-8d0b-8bbc5813c3a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:00.675923 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d07e6ad-d1da-4229-8d0b-8bbc5813c3a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:00.678085 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d07e6ad-d1da-4229-8d0b-8bbc5813c3a6 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:00.706824 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-3d07e6ad-d1da-4229-8d0b-8bbc5813c3a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:00.706824 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:00.706824 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:00.706824 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:00.706824 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:00.706824 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:00.706824 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:00.706824 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:00.706824 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:00.706824 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:00.706824 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:00.706824 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:00.706824 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:07:00.706824 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:07:00.706824 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:07:00.708078 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:07:00.708078 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:00.708078 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:00.708078 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:00.708078 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:00.708078 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:00.708078 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:00.708078 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:00.708078 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:00.708078 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:00.708078 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:00.708078 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:00.708078 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:00.708078 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:00.708078 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:00.708078 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:00.708078 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:00.709518 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:07:00.709518 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:07:00.709518 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:00.709518 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:00.709518 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 400/817] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:07:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:00.709518 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d07e6ad-d1da-4229-8d0b-8bbc5813c3a6 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:00.740467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e892f-b297-4742-8e0d-cee9f39d4eb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:00.743435 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e892f-b297-4742-8e0d-cee9f39d4eb4 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:00.745467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:00.841829 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:00.850923 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d07e6ad-d1da-4229-8d0b-8bbc5813c3a6 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:00.890285 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 418/818] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:07:00 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 06:07:00.894629 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32aeec9-1007-4173-a40f-ca22e39e122a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:00.895586 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32aeec9-1007-4173-a40f-ca22e39e122a None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:00.895976 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32aeec9-1007-4173-a40f-ca22e39e122a None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:00.896087 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb70a2-7d7d-4175-a94f-8da4f5a29c88 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:00.896354 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32aeec9-1007-4173-a40f-ca22e39e122a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:00.896494 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb70a2-7d7d-4175-a94f-8da4f5a29c88 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:00.896659 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb70a2-7d7d-4175-a94f-8da4f5a29c88 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:00.897032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32aeec9-1007-4173-a40f-ca22e39e122a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:00.898139 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 419/819] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:07:00.898422 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32aeec9-1007-4173-a40f-ca22e39e122a None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:00.903159 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb70a2-7d7d-4175-a94f-8da4f5a29c88 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:00.996404 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 401/820] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Jul 18 06:07:00 2022] POST /identity/v3/endpoints => generated 327 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:07:00.997708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32aeec9-1007-4173-a40f-ca22e39e122a None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:01.039192 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:01.049330 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb70a2-7d7d-4175-a94f-8da4f5a29c88 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:01.097129 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b0e1c-2d72-498f-a847-76d6de208826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:01.097882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b0e1c-2d72-498f-a847-76d6de208826 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:01.098277 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b0e1c-2d72-498f-a847-76d6de208826 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:01.098560 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b0e1c-2d72-498f-a847-76d6de208826 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:01.099584 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080b0e1c-2d72-498f-a847-76d6de208826 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:01.099584 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080b0e1c-2d72-498f-a847-76d6de208826 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:01.101087 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080b0e1c-2d72-498f-a847-76d6de208826 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:01.115511 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 420/821] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:07:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:01.116854 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672f34b-3b59-409b-a8b2-9b19b27f4fee None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:01.117146 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672f34b-3b59-409b-a8b2-9b19b27f4fee None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:01.117429 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672f34b-3b59-409b-a8b2-9b19b27f4fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:01.141244 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-4672f34b-3b59-409b-a8b2-9b19b27f4fee None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:01.143757 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4672f34b-3b59-409b-a8b2-9b19b27f4fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:01.166201 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32aeec9-1007-4173-a40f-ca22e39e122a None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:01.223842 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2293abec-0073-4a34-97af-cbb1bf2f7033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:01.224822 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293abec-0073-4a34-97af-cbb1bf2f7033 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:01.225111 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293abec-0073-4a34-97af-cbb1bf2f7033 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:01.225707 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293abec-0073-4a34-97af-cbb1bf2f7033 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:01.245376 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2293abec-0073-4a34-97af-cbb1bf2f7033 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:01.246947 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2293abec-0073-4a34-97af-cbb1bf2f7033 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:01.259347 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:01.263887 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 421/822] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:01 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:07:01.263887 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4672f34b-3b59-409b-a8b2-9b19b27f4fee None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:01.272178 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-2293abec-0073-4a34-97af-cbb1bf2f7033 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:01.272178 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:01.272178 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:01.272178 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:01.272178 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:01.272178 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:01.272178 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:01.272178 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:01.272178 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:01.272178 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:01.272178 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:01.272178 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:01.272178 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:07:01.272178 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:07:01.272178 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:07:01.275342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:07:01.275342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:01.275342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:01.275342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:01.275342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:01.275342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:01.275342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:01.275342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:01.275342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:01.275342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:01.275342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:01.275342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:01.275342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:01.275342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:01.275342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:01.275342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:01.275342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:01.276793 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:07:01.276793 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:07:01.276793 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:01.276793 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:01.276793 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 402/823] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:07:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:01.280268 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2293abec-0073-4a34-97af-cbb1bf2f7033 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:01.377558 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:01.392429 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4672f34b-3b59-409b-a8b2-9b19b27f4fee None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:01.395855 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2293abec-0073-4a34-97af-cbb1bf2f7033 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:01.432644 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0edf6-9a33-4008-be9a-4884be71d88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:01.433432 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c7f4a4-0e8f-4c5f-b2b3-49b7e0b2ab8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:01.433559 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0edf6-9a33-4008-be9a-4884be71d88d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:01.433837 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0edf6-9a33-4008-be9a-4884be71d88d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:01.434113 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c7f4a4-0e8f-4c5f-b2b3-49b7e0b2ab8f None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:01.434210 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0edf6-9a33-4008-be9a-4884be71d88d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:01.434398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c7f4a4-0e8f-4c5f-b2b3-49b7e0b2ab8f None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:01.434704 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c7f4a4-0e8f-4c5f-b2b3-49b7e0b2ab8f None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:01.435613 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c7f4a4-0e8f-4c5f-b2b3-49b7e0b2ab8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:01.435741 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c7f4a4-0e8f-4c5f-b2b3-49b7e0b2ab8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:01.437127 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c7f4a4-0e8f-4c5f-b2b3-49b7e0b2ab8f None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:01.442235 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0edf6-9a33-4008-be9a-4884be71d88d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:01.443694 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0edf6-9a33-4008-be9a-4884be71d88d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:01.453359 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 403/824] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:07:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:01.454227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c7f4a4-0e8f-4c5f-b2b3-49b7e0b2ab8f None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:01.460304 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-2ad0edf6-9a33-4008-be9a-4884be71d88d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 06:07:01.460304 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:01.460304 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:01.460304 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:01.460304 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:01.460304 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:01.460304 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:01.460304 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:01.460304 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:01.460304 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:01.460304 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:01.460304 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:01.460304 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 06:07:01.460304 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 06:07:01.460304 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 06:07:01.460304 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 06:07:01.461951 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:01.461951 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:01.461951 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 06:07:01.461951 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:07:01.461951 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 06:07:01.461951 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:07:01.461951 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:01.461951 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:01.461951 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:01.461951 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:01.461951 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:01.461951 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:01.461951 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:01.461951 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:01.461951 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:01.461951 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:01.461951 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:01.461951 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:01.463321 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:01.463321 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:01.463321 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 06:07:01.463321 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 06:07:01.463321 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 06:07:01.463321 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 06:07:01.463321 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 06:07:01.463321 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 06:07:01.463321 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 06:07:01.463321 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:01.463321 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 422/825] 10.222.0.40 () {56 vars in 1117 bytes} [Mon Jul 18 06:07:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:01.464018 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d5109-7837-4069-9287-e18cfa0eadd5 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:01.464018 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d5109-7837-4069-9287-e18cfa0eadd5 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:01.464474 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d5109-7837-4069-9287-e18cfa0eadd5 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:01.468464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 423/826] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:07:01.468464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d5109-7837-4069-9287-e18cfa0eadd5 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:01.526321 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:01.574533 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c7f4a4-0e8f-4c5f-b2b3-49b7e0b2ab8f None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:01.592709 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d5109-7837-4069-9287-e18cfa0eadd5 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:01.613420 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2450171f-0b6b-45a6-9c20-1ca2d83d8708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:01.614231 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2450171f-0b6b-45a6-9c20-1ca2d83d8708 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:01.614521 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2450171f-0b6b-45a6-9c20-1ca2d83d8708 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:01.614795 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2450171f-0b6b-45a6-9c20-1ca2d83d8708 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:01.615461 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2450171f-0b6b-45a6-9c20-1ca2d83d8708 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:01.616946 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2450171f-0b6b-45a6-9c20-1ca2d83d8708 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:01.629591 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6385757-ae58-46a4-a4a4-26fa418c71b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:01.630392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6385757-ae58-46a4-a4a4-26fa418c71b4 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:01.630683 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6385757-ae58-46a4-a4a4-26fa418c71b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:01.630796 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-2450171f-0b6b-45a6-9c20-1ca2d83d8708 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 06:07:01.630796 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:01.630796 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 06:07:01.630796 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 06:07:01.630796 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 06:07:01.630796 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 06:07:01.630796 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 06:07:01.630796 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:07:01.630796 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 06:07:01.630796 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:01.630796 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 06:07:01.630796 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:01.630796 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:01.630796 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:01.630796 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:01.630796 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:01.630796 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:01.631768 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:01.631768 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:01.631768 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:01.631768 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:01.631768 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:01.631768 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:01.631768 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 06:07:01.631768 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 06:07:01.631768 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 06:07:01.631768 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 06:07:01.631768 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:01.631768 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:01.631768 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:01.631768 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:01.631768 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:01.631768 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:01.632636 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:01.632636 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:01.632636 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:01.632636 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:01.632636 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:01.632636 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:01.632636 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:01.632636 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:01.632636 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:01.632636 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:01.632636 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 06:07:01.632636 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:07:01.632636 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 06:07:01.632636 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:01.633922 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6385757-ae58-46a4-a4a4-26fa418c71b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:01.634007 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 404/827] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 18 06:07:01 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:01.634579 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2450171f-0b6b-45a6-9c20-1ca2d83d8708 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:01.639677 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6385757-ae58-46a4-a4a4-26fa418c71b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:01.641180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6385757-ae58-46a4-a4a4-26fa418c71b4 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:01.656869 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-f6385757-ae58-46a4-a4a4-26fa418c71b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:07:01.656869 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:01.656869 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:01.656869 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:01.656869 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:01.656869 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:01.656869 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:01.656869 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:01.656869 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:01.656869 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:01.656869 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:01.656869 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:01.656869 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 06:07:01.656869 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 06:07:01.656869 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 06:07:01.658023 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 06:07:01.658023 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:01.658023 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:01.658023 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:01.658023 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:01.658023 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:01.658023 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:01.658023 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:01.658023 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:01.658023 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:01.658023 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:01.658023 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:01.658023 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:01.658023 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:01.658023 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:01.658023 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:01.658023 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:01.663995 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 06:07:01.663995 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 06:07:01.663995 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 06:07:01.663995 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 06:07:01.663995 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 06:07:01.663995 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 06:07:01.663995 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:07:01.663995 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:01.663995 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 424/828] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:07:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:01.663995 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9441e-c7a7-4be8-bc65-e66e6d92ffa3 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:01.663995 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9441e-c7a7-4be8-bc65-e66e6d92ffa3 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:01.664811 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9441e-c7a7-4be8-bc65-e66e6d92ffa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:01.678109 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-afc9441e-c7a7-4be8-bc65-e66e6d92ffa3 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:01.679370 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-afc9441e-c7a7-4be8-bc65-e66e6d92ffa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:01.681588 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:01.748844 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2450171f-0b6b-45a6-9c20-1ca2d83d8708 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:01.751748 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 425/829] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:01 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:07:01.751897 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc9441e-c7a7-4be8-bc65-e66e6d92ffa3 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:01.778445 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:01.789567 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88202da9-141b-456b-a92a-5ebe6e8551ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:01.790251 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88202da9-141b-456b-a92a-5ebe6e8551ee None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:01.790544 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88202da9-141b-456b-a92a-5ebe6e8551ee None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:01.790805 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88202da9-141b-456b-a92a-5ebe6e8551ee None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:01.791655 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88202da9-141b-456b-a92a-5ebe6e8551ee None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:01.791967 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88202da9-141b-456b-a92a-5ebe6e8551ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:01.793508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88202da9-141b-456b-a92a-5ebe6e8551ee None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:01.811476 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 405/830] 10.222.0.40 () {56 vars in 1149 bytes} [Mon Jul 18 06:07:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:01.812360 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88202da9-141b-456b-a92a-5ebe6e8551ee None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:01.856802 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc9441e-c7a7-4be8-bc65-e66e6d92ffa3 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:01.893413 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69669ad-dae6-48ff-9bfe-ec668247b19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:01.894547 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69669ad-dae6-48ff-9bfe-ec668247b19b None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:01.894835 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69669ad-dae6-48ff-9bfe-ec668247b19b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:01.895117 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69669ad-dae6-48ff-9bfe-ec668247b19b None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:01.895838 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69669ad-dae6-48ff-9bfe-ec668247b19b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:01.896149 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69669ad-dae6-48ff-9bfe-ec668247b19b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:01.897571 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69669ad-dae6-48ff-9bfe-ec668247b19b None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:01.906475 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 426/831] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 18 06:07:01 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:01.907589 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69669ad-dae6-48ff-9bfe-ec668247b19b None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:01.910775 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:01.920212 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88202da9-141b-456b-a92a-5ebe6e8551ee None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:01.955464 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea195e5-d2a2-4c89-af60-7d04dade29b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:01.956165 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea195e5-d2a2-4c89-af60-7d04dade29b0 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:01.956492 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea195e5-d2a2-4c89-af60-7d04dade29b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:01.956770 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea195e5-d2a2-4c89-af60-7d04dade29b0 None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:01.957573 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea195e5-d2a2-4c89-af60-7d04dade29b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:01.957881 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea195e5-d2a2-4c89-af60-7d04dade29b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:01.959313 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea195e5-d2a2-4c89-af60-7d04dade29b0 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:01.971767 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 406/832] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 18 06:07:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:01.972616 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea195e5-d2a2-4c89-af60-7d04dade29b0 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:02.013099 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69669ad-dae6-48ff-9bfe-ec668247b19b None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:02.017993 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:02.051933 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3673a3bc-8a5b-4837-8f4a-53a5dc3b55ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:02.052671 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673a3bc-8a5b-4837-8f4a-53a5dc3b55ad None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:02.052967 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673a3bc-8a5b-4837-8f4a-53a5dc3b55ad None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:02.053242 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673a3bc-8a5b-4837-8f4a-53a5dc3b55ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:02.064613 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3673a3bc-8a5b-4837-8f4a-53a5dc3b55ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:07:02.072915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-3673a3bc-8a5b-4837-8f4a-53a5dc3b55ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:07:02.072915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:02.072915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:02.072915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:02.072915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:02.072915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:02.072915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:02.072915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:02.072915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:02.072915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:02.072915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:02.072915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:02.072915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:07:02.072915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:07:02.072915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:07:02.072915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:07:02.073910 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:07:02.073910 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:07:02.073910 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:07:02.073910 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:07:02.073910 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:02.073910 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:02.073910 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:02.073910 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:02.073910 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:02.073910 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:02.073910 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:02.073910 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:02.073910 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:02.073910 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:02.073910 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:02.073910 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:02.074936 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:02.074936 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:02.074936 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:02.074936 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:02.074936 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:07:02.074936 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:07:02.074936 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:07:02.074936 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:07:02.074936 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:07:02.074936 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:07:02.074936 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:07:02.074936 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:02.074936 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 427/833] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 18 06:07:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:02.076340 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3673a3bc-8a5b-4837-8f4a-53a5dc3b55ad None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:02.079393 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea195e5-d2a2-4c89-af60-7d04dade29b0 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:02.115672 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe6519-770b-42a0-8589-aab5907ff31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:02.116510 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe6519-770b-42a0-8589-aab5907ff31c None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:02.116802 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe6519-770b-42a0-8589-aab5907ff31c None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:02.117080 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe6519-770b-42a0-8589-aab5907ff31c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:02.129024 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfe6519-770b-42a0-8589-aab5907ff31c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:02.130619 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfe6519-770b-42a0-8589-aab5907ff31c None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:02.141109 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:02.148534 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-ecfe6519-770b-42a0-8589-aab5907ff31c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:02.148534 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:02.148534 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:02.148534 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:02.148534 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:02.148534 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:02.148534 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:02.148534 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:02.148534 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:02.148534 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:02.148534 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:02.148534 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:02.148534 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:07:02.148534 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:07:02.148534 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:07:02.156019 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:07:02.156019 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:02.156019 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:02.156019 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:02.156019 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:02.156019 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:02.156019 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:02.156019 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:02.156019 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:02.156019 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:02.156019 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:02.156019 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:02.156019 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:02.156019 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:02.156019 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:02.156019 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:02.156019 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:02.157159 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:07:02.157159 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:07:02.157159 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:02.157159 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:02.157159 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 407/834] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:07:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:02.157159 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe6519-770b-42a0-8589-aab5907ff31c None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:02.182123 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3673a3bc-8a5b-4837-8f4a-53a5dc3b55ad None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:02.216672 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e57e29-8c65-4bbf-990a-bfbc3a575349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:02.217323 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e57e29-8c65-4bbf-990a-bfbc3a575349 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:02.217605 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e57e29-8c65-4bbf-990a-bfbc3a575349 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:02.217884 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e57e29-8c65-4bbf-990a-bfbc3a575349 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:02.218607 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e57e29-8c65-4bbf-990a-bfbc3a575349 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:02.218900 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e57e29-8c65-4bbf-990a-bfbc3a575349 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:02.220279 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e57e29-8c65-4bbf-990a-bfbc3a575349 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:02.225046 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 428/835] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 18 06:07:02 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:02.225664 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e57e29-8c65-4bbf-990a-bfbc3a575349 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:02.264872 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:02.272939 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe6519-770b-42a0-8589-aab5907ff31c None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:02.318574 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b882b773-1d33-403a-8503-85f06d01072b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:02.319187 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882b773-1d33-403a-8503-85f06d01072b None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:02.319477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882b773-1d33-403a-8503-85f06d01072b None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:02.319758 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882b773-1d33-403a-8503-85f06d01072b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:02.320698 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b882b773-1d33-403a-8503-85f06d01072b None admin] RBAC: Adding query filter params (role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc, scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a, user.id=f92d948810ea4241810473506b654f7a) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:02.321021 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b882b773-1d33-403a-8503-85f06d01072b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:02.322496 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b882b773-1d33-403a-8503-85f06d01072b None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:02.329783 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 408/836] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:07:02 2022] GET /identity/v3/role_assignments?user.id=f92d948810ea4241810473506b654f7a&scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a&role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:02.330754 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b882b773-1d33-403a-8503-85f06d01072b None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:02.377343 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e57e29-8c65-4bbf-990a-bfbc3a575349 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:02.419554 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:02.428250 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b83ea-57e3-4d85-992c-72112f362889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:02.429412 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b83ea-57e3-4d85-992c-72112f362889 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:02.436259 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b83ea-57e3-4d85-992c-72112f362889 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:02.436259 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b83ea-57e3-4d85-992c-72112f362889 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:02.436259 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b83ea-57e3-4d85-992c-72112f362889 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:02.436259 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b83ea-57e3-4d85-992c-72112f362889 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:02.436259 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b83ea-57e3-4d85-992c-72112f362889 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:02.446896 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 429/837] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:07:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:02.453075 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b83ea-57e3-4d85-992c-72112f362889 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:02.482252 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b882b773-1d33-403a-8503-85f06d01072b None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:02.527885 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3646b6-fbd9-4846-822d-1a4107ddc520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:02.528620 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3646b6-fbd9-4846-822d-1a4107ddc520 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:02.528900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3646b6-fbd9-4846-822d-1a4107ddc520 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:02.529374 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3646b6-fbd9-4846-822d-1a4107ddc520 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:02.530529 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3646b6-fbd9-4846-822d-1a4107ddc520 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:02.530529 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3646b6-fbd9-4846-822d-1a4107ddc520 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:02.531732 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3646b6-fbd9-4846-822d-1a4107ddc520 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:02.546315 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 409/838] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:07:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:02.552184 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3646b6-fbd9-4846-822d-1a4107ddc520 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:02.579661 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:02.590262 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b83ea-57e3-4d85-992c-72112f362889 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:02.629656 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97465be7-f2f7-4ed3-bd95-886ebaccbf5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:02.630556 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97465be7-f2f7-4ed3-bd95-886ebaccbf5f None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:02.630858 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97465be7-f2f7-4ed3-bd95-886ebaccbf5f None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:02.635331 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97465be7-f2f7-4ed3-bd95-886ebaccbf5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:02.643065 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97465be7-f2f7-4ed3-bd95-886ebaccbf5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:02.644613 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97465be7-f2f7-4ed3-bd95-886ebaccbf5f None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:02.664177 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-97465be7-f2f7-4ed3-bd95-886ebaccbf5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:02.664177 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:02.664177 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:02.664177 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:02.664177 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:02.664177 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:02.664177 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:02.664177 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:02.664177 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:02.664177 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:02.664177 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:02.664177 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:02.664177 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:07:02.664177 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:07:02.664177 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:07:02.665407 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:07:02.665407 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:02.665407 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:02.665407 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:02.665407 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:02.665407 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:02.665407 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:02.665407 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:02.665407 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:02.665407 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:02.665407 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:02.665407 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:02.665407 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:02.665407 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:02.665407 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:02.665407 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:02.665407 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:02.666605 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:07:02.666605 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:07:02.666605 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:02.666605 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:02.666605 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 430/839] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:07:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:07:02.671326 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97465be7-f2f7-4ed3-bd95-886ebaccbf5f None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:02.700907 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3646b6-fbd9-4846-822d-1a4107ddc520 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:02.722621 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:02.738941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c75d90-6179-475e-b8d3-7adabec88ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:02.740037 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c75d90-6179-475e-b8d3-7adabec88ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:02.740594 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c75d90-6179-475e-b8d3-7adabec88ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:02.741079 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c75d90-6179-475e-b8d3-7adabec88ac0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:02.751171 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c75d90-6179-475e-b8d3-7adabec88ac0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:02.752656 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c75d90-6179-475e-b8d3-7adabec88ac0 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:02.769559 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-69c75d90-6179-475e-b8d3-7adabec88ac0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:07:02.769559 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:02.769559 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:02.769559 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:02.769559 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:02.769559 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:02.769559 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:02.769559 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:02.769559 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:02.769559 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:02.769559 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:02.769559 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:02.769559 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 06:07:02.769559 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 06:07:02.769559 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 06:07:02.770647 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 06:07:02.770647 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:02.770647 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:02.770647 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:02.770647 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:02.770647 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:02.770647 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:02.770647 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:02.770647 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:02.770647 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:02.770647 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:02.770647 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:02.770647 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:02.770647 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:02.770647 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:02.770647 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:02.770647 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:02.771577 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 06:07:02.771577 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 06:07:02.771577 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 06:07:02.771577 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 06:07:02.771577 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 06:07:02.771577 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 06:07:02.771577 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:07:02.771577 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:02.771577 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 410/840] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:07:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:02.777494 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c75d90-6179-475e-b8d3-7adabec88ac0 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:02.802360 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97465be7-f2f7-4ed3-bd95-886ebaccbf5f None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:02.841310 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bac6f0-6786-44e1-b5bc-2d64999921df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:02.842318 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bac6f0-6786-44e1-b5bc-2d64999921df None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:02.842801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bac6f0-6786-44e1-b5bc-2d64999921df None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:02.843318 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bac6f0-6786-44e1-b5bc-2d64999921df None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:02.844385 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bac6f0-6786-44e1-b5bc-2d64999921df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:02.844921 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bac6f0-6786-44e1-b5bc-2d64999921df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:02.846579 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bac6f0-6786-44e1-b5bc-2d64999921df None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:02.862976 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 431/841] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:07:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:07:02.868994 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bac6f0-6786-44e1-b5bc-2d64999921df None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:02.869354 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:02.893716 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c75d90-6179-475e-b8d3-7adabec88ac0 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:02.932192 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b8d3b-8c12-46d6-afaa-6b76f9af0f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:02.933621 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b8d3b-8c12-46d6-afaa-6b76f9af0f9b None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:02.934055 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b8d3b-8c12-46d6-afaa-6b76f9af0f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:02.934483 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b8d3b-8c12-46d6-afaa-6b76f9af0f9b None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:02.935432 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b8d3b-8c12-46d6-afaa-6b76f9af0f9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:02.935889 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b8d3b-8c12-46d6-afaa-6b76f9af0f9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:02.937526 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b8d3b-8c12-46d6-afaa-6b76f9af0f9b None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:02.951992 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 411/842] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 18 06:07:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:02.959479 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b8d3b-8c12-46d6-afaa-6b76f9af0f9b None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:02.991506 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:02.995795 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bac6f0-6786-44e1-b5bc-2d64999921df None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:03.038991 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6e522-7aab-40ce-977f-dd2887873c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:03.040142 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6e522-7aab-40ce-977f-dd2887873c2e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:03.040621 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6e522-7aab-40ce-977f-dd2887873c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:03.041041 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6e522-7aab-40ce-977f-dd2887873c2e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:03.050025 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee6e522-7aab-40ce-977f-dd2887873c2e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:03.051807 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee6e522-7aab-40ce-977f-dd2887873c2e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:03.067760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-1ee6e522-7aab-40ce-977f-dd2887873c2e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 06:07:03.067760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:03.067760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:03.067760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:03.067760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:03.067760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:03.067760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:03.067760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:03.067760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:03.067760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:03.067760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:03.067760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:03.067760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 06:07:03.067760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 06:07:03.067760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 06:07:03.067760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 06:07:03.068958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:03.068958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:03.068958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 06:07:03.068958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:07:03.068958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 06:07:03.068958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:07:03.068958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:03.068958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:03.068958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:03.068958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:03.068958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:03.068958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:03.068958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:03.068958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:03.068958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:03.068958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:03.068958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:03.068958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:03.070157 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:03.070157 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:03.070157 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 06:07:03.070157 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 06:07:03.070157 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 06:07:03.070157 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 06:07:03.070157 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 06:07:03.070157 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 06:07:03.070157 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 06:07:03.070157 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:03.077682 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 432/843] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 18 06:07:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:07:03.078769 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6e522-7aab-40ce-977f-dd2887873c2e None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:03.082482 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b8d3b-8c12-46d6-afaa-6b76f9af0f9b None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:03.120622 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176fc422-48b4-41bb-931e-5d97b8610021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:03.121544 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176fc422-48b4-41bb-931e-5d97b8610021 None admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:03.121899 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176fc422-48b4-41bb-931e-5d97b8610021 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:03.122247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176fc422-48b4-41bb-931e-5d97b8610021 None admin] PATH_INFO: `/v3/projects/8ed50a8a3e184bde9269d7b0d9573d4a/users/b84dde57ff8e47419866377a819bea21/roles/ab10744ff77646eab23a769ab89c74aa` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:03.131430 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:03.143514 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176fc422-48b4-41bb-931e-5d97b8610021 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ed50a8a3e184bde9269d7b0d9573d4a, user_id=b84dde57ff8e47419866377a819bea21, role_id=ab10744ff77646eab23a769ab89c74aa)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:03.146013 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176fc422-48b4-41bb-931e-5d97b8610021 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:03.197401 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6e522-7aab-40ce-977f-dd2887873c2e None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:03.229624 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 412/844] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 18 06:07:02 2022] PUT /identity/v3/projects/8ed50a8a3e184bde9269d7b0d9573d4a/users/b84dde57ff8e47419866377a819bea21/roles/ab10744ff77646eab23a769ab89c74aa => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:07:03.242758 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1658e0-4960-4928-957e-813d9cc976b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:03.247156 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1658e0-4960-4928-957e-813d9cc976b7 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:03.247156 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1658e0-4960-4928-957e-813d9cc976b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:03.247156 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1658e0-4960-4928-957e-813d9cc976b7 None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:03.247156 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1658e0-4960-4928-957e-813d9cc976b7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:03.247156 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1658e0-4960-4928-957e-813d9cc976b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:03.247156 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1658e0-4960-4928-957e-813d9cc976b7 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:03.270909 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 433/845] 10.222.0.40 () {56 vars in 1135 bytes} [Mon Jul 18 06:07:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:03.277060 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176fc422-48b4-41bb-931e-5d97b8610021 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:03.411439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176fc422-48b4-41bb-931e-5d97b8610021 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:03.450137 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd36ae-6d01-4cf0-89b3-32852872884e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:03.450789 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd36ae-6d01-4cf0-89b3-32852872884e None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:03.451079 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd36ae-6d01-4cf0-89b3-32852872884e None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:03.451365 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd36ae-6d01-4cf0-89b3-32852872884e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:03.516190 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:03.529718 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cd36ae-6d01-4cf0-89b3-32852872884e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:03.531168 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cd36ae-6d01-4cf0-89b3-32852872884e None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:03.639955 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 413/846] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:07:03 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 06:07:03.645699 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1658e0-4960-4928-957e-813d9cc976b7 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:03.671044 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:03.774324 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1658e0-4960-4928-957e-813d9cc976b7 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:03.812604 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0eef53-5f5e-4840-8c37-590811972b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:03.813539 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0eef53-5f5e-4840-8c37-590811972b44 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:03.813827 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0eef53-5f5e-4840-8c37-590811972b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:03.814098 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0eef53-5f5e-4840-8c37-590811972b44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:03.825945 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0eef53-5f5e-4840-8c37-590811972b44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:03.827413 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0eef53-5f5e-4840-8c37-590811972b44 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:03.852039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-9f0eef53-5f5e-4840-8c37-590811972b44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:03.852039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:03.852039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:03.852039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:03.852039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:03.852039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:03.852039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:03.852039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:03.852039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:03.852039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:03.852039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:03.852039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:03.852039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:07:03.852039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:07:03.852039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:07:03.856821 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:07:03.856821 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:03.856821 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:03.856821 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:03.856821 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:03.856821 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:03.856821 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:03.856821 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:03.856821 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:03.856821 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:03.856821 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:03.856821 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:03.856821 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:03.856821 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:03.856821 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:03.856821 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:03.856821 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:03.857754 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:07:03.857754 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:07:03.857754 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:03.857754 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:03.857754 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 434/847] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:07:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:03.859534 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd36ae-6d01-4cf0-89b3-32852872884e None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:03.956556 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:03.969627 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd36ae-6d01-4cf0-89b3-32852872884e None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:04.006437 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe762d2-4e69-4a0a-8c9e-5f9f99583f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:04.007206 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe762d2-4e69-4a0a-8c9e-5f9f99583f42 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:04.007673 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe762d2-4e69-4a0a-8c9e-5f9f99583f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:04.008014 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe762d2-4e69-4a0a-8c9e-5f9f99583f42 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:04.009222 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe762d2-4e69-4a0a-8c9e-5f9f99583f42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:04.009553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe762d2-4e69-4a0a-8c9e-5f9f99583f42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:04.010942 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe762d2-4e69-4a0a-8c9e-5f9f99583f42 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:04.023677 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 414/848] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:07:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:04.029037 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0eef53-5f5e-4840-8c37-590811972b44 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:04.104800 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:04.138051 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0eef53-5f5e-4840-8c37-590811972b44 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:04.177977 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a16eee5-d6d9-4a14-b1e3-93252500a7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:04.178838 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16eee5-d6d9-4a14-b1e3-93252500a7d2 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:04.179130 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16eee5-d6d9-4a14-b1e3-93252500a7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:04.179409 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16eee5-d6d9-4a14-b1e3-93252500a7d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:04.187703 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a16eee5-d6d9-4a14-b1e3-93252500a7d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:04.189206 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a16eee5-d6d9-4a14-b1e3-93252500a7d2 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:04.203915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-6a16eee5-d6d9-4a14-b1e3-93252500a7d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:07:04.203915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:04.203915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:04.203915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:04.203915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:04.203915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:04.203915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:04.203915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:04.203915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:04.203915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:04.203915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:04.203915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:04.203915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 06:07:04.203915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 06:07:04.203915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 06:07:04.205094 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 06:07:04.205094 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:04.205094 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:04.205094 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:04.205094 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:04.205094 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:04.205094 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:04.205094 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:04.205094 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:04.205094 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:04.205094 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:04.205094 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:04.205094 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:04.205094 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:04.205094 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:04.205094 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:04.205094 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:04.206317 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 06:07:04.206317 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 06:07:04.206317 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 06:07:04.206317 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 06:07:04.206317 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 06:07:04.206317 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 06:07:04.206317 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:07:04.206317 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:04.206317 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 435/849] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:07:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:04.211688 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe762d2-4e69-4a0a-8c9e-5f9f99583f42 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:04.328521 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe762d2-4e69-4a0a-8c9e-5f9f99583f42 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:04.365951 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8b9171-8a11-4ac3-9b7f-6add04794415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:04.366613 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8b9171-8a11-4ac3-9b7f-6add04794415 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:04.366897 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8b9171-8a11-4ac3-9b7f-6add04794415 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:04.367173 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8b9171-8a11-4ac3-9b7f-6add04794415 None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:04.368265 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8b9171-8a11-4ac3-9b7f-6add04794415 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:04.368612 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8b9171-8a11-4ac3-9b7f-6add04794415 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:04.370052 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8b9171-8a11-4ac3-9b7f-6add04794415 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:04.383343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 415/850] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 18 06:07:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:04.389278 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a16eee5-d6d9-4a14-b1e3-93252500a7d2 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:04.449392 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eee93e-b5f8-48ca-bc62-b09a7bbc4a26 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:04.449855 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eee93e-b5f8-48ca-bc62-b09a7bbc4a26 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:04.450227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eee93e-b5f8-48ca-bc62-b09a7bbc4a26 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:04.451925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 416/851] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:07:04.457120 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab76fb2-29c3-485e-ab73-96c3f58df529 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:04.457402 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab76fb2-29c3-485e-ab73-96c3f58df529 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:04.457781 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab76fb2-29c3-485e-ab73-96c3f58df529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:04.476084 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-4ab76fb2-29c3-485e-ab73-96c3f58df529 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:04.477857 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4ab76fb2-29c3-485e-ab73-96c3f58df529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:04.480455 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:04.505674 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a16eee5-d6d9-4a14-b1e3-93252500a7d2 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:04.543406 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:04.552371 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa70a9-1064-41c7-9c88-3f1af9684cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:04.553084 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa70a9-1064-41c7-9c88-3f1af9684cbf None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:04.553373 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa70a9-1064-41c7-9c88-3f1af9684cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:04.553740 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa70a9-1064-41c7-9c88-3f1af9684cbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:04.554660 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aa70a9-1064-41c7-9c88-3f1af9684cbf None admin] RBAC: Adding query filter params (role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc, scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a, user.id=0681f5fbea6d415c9ad2f3afae256a5c) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:04.555526 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aa70a9-1064-41c7-9c88-3f1af9684cbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:04.557037 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aa70a9-1064-41c7-9c88-3f1af9684cbf None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:04.560860 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 417/852] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:04 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:07:04.564233 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 436/853] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:07:04 2022] GET /identity/v3/role_assignments?user.id=0681f5fbea6d415c9ad2f3afae256a5c&scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a&role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:04.580473 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab76fb2-29c3-485e-ab73-96c3f58df529 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:04.622429 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb210b-0fb7-400f-95ba-a148c809c499 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:04.622794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb210b-0fb7-400f-95ba-a148c809c499 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:04.622999 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb210b-0fb7-400f-95ba-a148c809c499 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:04.624434 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 437/854] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:07:04.631427 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ec164-abbc-4803-95c4-42865b389c52 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:04.631759 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ec164-abbc-4803-95c4-42865b389c52 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:04.631990 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ec164-abbc-4803-95c4-42865b389c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:04.651461 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-299ec164-abbc-4803-95c4-42865b389c52 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:04.655571 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-299ec164-abbc-4803-95c4-42865b389c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:04.741021 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab76fb2-29c3-485e-ab73-96c3f58df529 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:04.752681 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 438/855] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:04 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 06:07:04.777615 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ec164-abbc-4803-95c4-42865b389c52 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:04.788408 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e995ad4-0162-41d2-ad66-5c1f8b12e8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:04.789115 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e995ad4-0162-41d2-ad66-5c1f8b12e8dc None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:04.790054 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e995ad4-0162-41d2-ad66-5c1f8b12e8dc None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:04.790681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e995ad4-0162-41d2-ad66-5c1f8b12e8dc None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:04.791254 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e995ad4-0162-41d2-ad66-5c1f8b12e8dc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:04.792672 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e995ad4-0162-41d2-ad66-5c1f8b12e8dc None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:04.893018 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ec164-abbc-4803-95c4-42865b389c52 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:04.915542 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:04.921885 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:04.929483 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 418/856] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Jul 18 06:07:04 2022] POST /identity/v3/services => generated 251 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:07:04.942111 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ab76a-0217-4614-a58a-4764655b7b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:04.943310 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ab76a-0217-4614-a58a-4764655b7b2e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:04.943795 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ab76a-0217-4614-a58a-4764655b7b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:04.944217 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ab76a-0217-4614-a58a-4764655b7b2e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:04.945128 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9ab76a-0217-4614-a58a-4764655b7b2e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:04.947277 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9ab76a-0217-4614-a58a-4764655b7b2e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:04.961205 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-3d9ab76a-0217-4614-a58a-4764655b7b2e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 06:07:04.961205 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:04.961205 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 06:07:04.961205 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 06:07:04.961205 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 06:07:04.961205 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 06:07:04.961205 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 06:07:04.961205 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:07:04.961205 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 06:07:04.961205 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:04.961205 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 06:07:04.961205 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:04.961205 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:04.961205 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:04.961205 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:04.961205 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:04.962662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:04.962662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:04.962662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:04.962662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:04.962662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:04.962662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:04.962662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:04.962662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 06:07:04.962662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 06:07:04.962662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 06:07:04.962662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 06:07:04.962662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:04.962662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:04.962662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:04.962662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:04.962662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:04.963688 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:04.963688 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:04.963688 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:04.963688 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:04.963688 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:04.963688 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:04.963688 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:04.963688 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:04.963688 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:04.963688 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:04.963688 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:04.963688 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 06:07:04.963688 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:07:04.963688 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 06:07:04.963688 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:04.966480 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 439/857] 10.222.0.40 () {56 vars in 1132 bytes} [Mon Jul 18 06:07:04 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:07:04.976476 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e995ad4-0162-41d2-ad66-5c1f8b12e8dc None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:05.210510 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e995ad4-0162-41d2-ad66-5c1f8b12e8dc None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:05.250274 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482adcf-ecb9-4a49-a19d-e83e7016ae69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:05.250945 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482adcf-ecb9-4a49-a19d-e83e7016ae69 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:05.251228 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482adcf-ecb9-4a49-a19d-e83e7016ae69 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:05.251501 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482adcf-ecb9-4a49-a19d-e83e7016ae69 None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:05.252217 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482adcf-ecb9-4a49-a19d-e83e7016ae69 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:05.252554 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482adcf-ecb9-4a49-a19d-e83e7016ae69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:05.253930 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482adcf-ecb9-4a49-a19d-e83e7016ae69 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:05.259792 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 419/858] 10.222.0.40 () {56 vars in 1126 bytes} [Mon Jul 18 06:07:04 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:05.265635 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ab76a-0217-4614-a58a-4764655b7b2e None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:05.379033 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ab76a-0217-4614-a58a-4764655b7b2e None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:05.416586 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a3c09-d31c-491d-b77c-d49353833c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:05.417266 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a3c09-d31c-491d-b77c-d49353833c28 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:05.417550 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a3c09-d31c-491d-b77c-d49353833c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:05.417824 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a3c09-d31c-491d-b77c-d49353833c28 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:05.418538 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a3c09-d31c-491d-b77c-d49353833c28 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:05.418847 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a3c09-d31c-491d-b77c-d49353833c28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:05.420222 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a3c09-d31c-491d-b77c-d49353833c28 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:05.425710 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 440/859] 10.222.0.40 () {56 vars in 1126 bytes} [Mon Jul 18 06:07:05 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:05.543379 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e761eccd-7a47-4234-bf45-17ac98187452 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:05.544021 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e761eccd-7a47-4234-bf45-17ac98187452 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:05.544543 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e761eccd-7a47-4234-bf45-17ac98187452 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:05.546464 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 420/860] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:07:05.552270 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9d93c-f1eb-40f1-8c55-0b7cfe39fd7b None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:05.552661 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9d93c-f1eb-40f1-8c55-0b7cfe39fd7b None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:05.552950 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9d93c-f1eb-40f1-8c55-0b7cfe39fd7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:05.572374 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-65b9d93c-f1eb-40f1-8c55-0b7cfe39fd7b None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:05.573645 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-65b9d93c-f1eb-40f1-8c55-0b7cfe39fd7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:05.626330 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:05.650866 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 441/861] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:05 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 06:07:05.676089 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e761eccd-7a47-4234-bf45-17ac98187452 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:05.760939 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:05.808644 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e761eccd-7a47-4234-bf45-17ac98187452 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:05.850489 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef82e12c-a922-4cf7-bf28-4e20e08b01ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:05.851389 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82e12c-a922-4cf7-bf28-4e20e08b01ce None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:05.852311 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82e12c-a922-4cf7-bf28-4e20e08b01ce None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:05.852616 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82e12c-a922-4cf7-bf28-4e20e08b01ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:05.865049 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef82e12c-a922-4cf7-bf28-4e20e08b01ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:07:05.873329 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-ef82e12c-a922-4cf7-bf28-4e20e08b01ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:07:05.873329 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:05.873329 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:05.873329 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:05.873329 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:05.873329 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:05.873329 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:05.873329 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:05.873329 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:05.873329 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:05.873329 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:05.873329 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:05.873329 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:07:05.873329 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:07:05.873329 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:07:05.873329 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:07:05.874321 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:07:05.874321 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:07:05.874321 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:07:05.874321 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:07:05.874321 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:05.874321 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:05.874321 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:05.874321 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:05.874321 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:05.874321 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:05.874321 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:05.874321 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:05.874321 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:05.874321 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:05.874321 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:05.874321 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:05.875478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:05.875478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:05.875478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:05.875478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:05.875478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:07:05.875478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:07:05.875478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:07:05.875478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:07:05.875478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:07:05.875478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:07:05.875478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:07:05.875478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:05.875478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 421/862] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 18 06:07:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:05.882664 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b9d93c-f1eb-40f1-8c55-0b7cfe39fd7b None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:05.992835 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b9d93c-f1eb-40f1-8c55-0b7cfe39fd7b None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:06.030244 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab7bc9-da33-4bad-9b58-f734b72a4984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:06.031399 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab7bc9-da33-4bad-9b58-f734b72a4984 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:06.031895 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab7bc9-da33-4bad-9b58-f734b72a4984 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:06.032376 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab7bc9-da33-4bad-9b58-f734b72a4984 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:06.033377 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ab7bc9-da33-4bad-9b58-f734b72a4984 None admin] RBAC: Adding query filter params (name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:06.033889 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ab7bc9-da33-4bad-9b58-f734b72a4984 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:06.035491 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ab7bc9-da33-4bad-9b58-f734b72a4984 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:06.051060 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 442/863] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 18 06:07:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:06.052861 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef82e12c-a922-4cf7-bf28-4e20e08b01ce None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:06.169713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef82e12c-a922-4cf7-bf28-4e20e08b01ce None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:06.196773 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:06.209588 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea17fd-0dfc-4faa-892c-b04a4f4b7628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:06.211147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea17fd-0dfc-4faa-892c-b04a4f4b7628 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:06.211147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea17fd-0dfc-4faa-892c-b04a4f4b7628 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:06.211147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea17fd-0dfc-4faa-892c-b04a4f4b7628 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:06.222504 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aea17fd-0dfc-4faa-892c-b04a4f4b7628 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:06.223953 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aea17fd-0dfc-4faa-892c-b04a4f4b7628 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:06.243706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-6aea17fd-0dfc-4faa-892c-b04a4f4b7628 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:06.243706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:06.243706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:06.243706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:06.243706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:06.243706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:06.243706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:06.243706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:06.243706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:06.243706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:06.243706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:06.243706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:06.243706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:07:06.243706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:07:06.243706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:07:06.244779 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:07:06.244779 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:06.244779 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:06.244779 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:06.244779 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:06.244779 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:06.244779 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:06.244779 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:06.244779 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:06.244779 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:06.244779 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:06.244779 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:06.244779 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:06.244779 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:06.244779 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:06.244779 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:06.244779 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:06.245734 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:07:06.245734 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:07:06.245734 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:06.245734 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:06.246552 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 422/864] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:07:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:06.252751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab7bc9-da33-4bad-9b58-f734b72a4984 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:06.300581 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:06.369333 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab7bc9-da33-4bad-9b58-f734b72a4984 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:06.407340 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003bf5ae-0de1-42b1-88e1-be751e13c3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:06.408502 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003bf5ae-0de1-42b1-88e1-be751e13c3bf None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:06.409030 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003bf5ae-0de1-42b1-88e1-be751e13c3bf None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:06.409529 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003bf5ae-0de1-42b1-88e1-be751e13c3bf None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:06.410524 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003bf5ae-0de1-42b1-88e1-be751e13c3bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:06.411337 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003bf5ae-0de1-42b1-88e1-be751e13c3bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:06.413006 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003bf5ae-0de1-42b1-88e1-be751e13c3bf None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:06.432683 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 443/865] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:07:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:06.436037 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea17fd-0dfc-4faa-892c-b04a4f4b7628 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:06.607271 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea17fd-0dfc-4faa-892c-b04a4f4b7628 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:06.667230 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16dce2-4fd7-4056-a778-594c5452f7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:06.668567 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16dce2-4fd7-4056-a778-594c5452f7ff None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:06.669018 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16dce2-4fd7-4056-a778-594c5452f7ff None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:06.669457 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16dce2-4fd7-4056-a778-594c5452f7ff None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:06.684651 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca16dce2-4fd7-4056-a778-594c5452f7ff None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:06.691065 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca16dce2-4fd7-4056-a778-594c5452f7ff None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:06.716146 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-ca16dce2-4fd7-4056-a778-594c5452f7ff None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 06:07:06.716146 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:06.716146 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:06.716146 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:06.716146 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:06.716146 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:06.716146 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:06.716146 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:06.716146 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:06.716146 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:06.716146 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:06.716146 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:06.716146 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 06:07:06.716146 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 06:07:06.716146 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 06:07:06.716146 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 06:07:06.717216 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:06.717216 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:06.717216 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 06:07:06.717216 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:07:06.717216 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 06:07:06.717216 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:07:06.717216 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:06.717216 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:06.717216 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:06.717216 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:06.717216 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:06.717216 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:06.717216 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:06.717216 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:06.717216 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:06.717216 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:06.717216 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:06.717216 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:06.718222 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:06.718222 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:06.718222 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 06:07:06.718222 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 06:07:06.718222 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 06:07:06.718222 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 06:07:06.718222 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 06:07:06.718222 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 06:07:06.718222 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 06:07:06.718222 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:06.718979 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 423/866] 10.222.0.40 () {56 vars in 1117 bytes} [Mon Jul 18 06:07:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:07:06.725638 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003bf5ae-0de1-42b1-88e1-be751e13c3bf None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:06.853293 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:06.856667 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003bf5ae-0de1-42b1-88e1-be751e13c3bf None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:06.912687 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dacfd77-0c8c-461c-b79f-1eb667d6940a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:06.913904 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacfd77-0c8c-461c-b79f-1eb667d6940a None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:06.914938 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacfd77-0c8c-461c-b79f-1eb667d6940a None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:06.915400 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacfd77-0c8c-461c-b79f-1eb667d6940a None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:06.916994 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dacfd77-0c8c-461c-b79f-1eb667d6940a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:06.917551 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dacfd77-0c8c-461c-b79f-1eb667d6940a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:06.920295 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dacfd77-0c8c-461c-b79f-1eb667d6940a None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:06.949569 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 444/867] 10.222.0.40 () {56 vars in 1149 bytes} [Mon Jul 18 06:07:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:06.955364 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16dce2-4fd7-4056-a778-594c5452f7ff None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:06.978433 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc1abf-1a56-40e6-9779-4619ed902db1 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:06.978926 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc1abf-1a56-40e6-9779-4619ed902db1 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:06.979436 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc1abf-1a56-40e6-9779-4619ed902db1 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:06.981689 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 445/868] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:07:06.994107 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46976d-6910-4f40-aa6a-a83189d79231 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:06.994456 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46976d-6910-4f40-aa6a-a83189d79231 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:06.994744 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46976d-6910-4f40-aa6a-a83189d79231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:07.000750 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:07.013924 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-be46976d-6910-4f40-aa6a-a83189d79231 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:07.015178 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-be46976d-6910-4f40-aa6a-a83189d79231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:07.102790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16dce2-4fd7-4056-a778-594c5452f7ff None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:07.112503 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 446/869] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:06 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:07:07.131793 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be46976d-6910-4f40-aa6a-a83189d79231 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:07.144690 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e414831a-0228-445c-b551-61a2fdd2091c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:07.146201 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414831a-0228-445c-b551-61a2fdd2091c None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:07.147058 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414831a-0228-445c-b551-61a2fdd2091c None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:07.147541 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414831a-0228-445c-b551-61a2fdd2091c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:07.240515 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e414831a-0228-445c-b551-61a2fdd2091c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:07.243355 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e414831a-0228-445c-b551-61a2fdd2091c None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:07.256714 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:07.272275 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be46976d-6910-4f40-aa6a-a83189d79231 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:07.285288 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:07.320950 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4594b-6b0d-4dba-bfaf-033f6bca8f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:07.321779 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4594b-6b0d-4dba-bfaf-033f6bca8f92 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:07.322066 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4594b-6b0d-4dba-bfaf-033f6bca8f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:07.322348 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4594b-6b0d-4dba-bfaf-033f6bca8f92 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:07.323033 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b4594b-6b0d-4dba-bfaf-033f6bca8f92 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:07.324700 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b4594b-6b0d-4dba-bfaf-033f6bca8f92 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:07.338039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-84b4594b-6b0d-4dba-bfaf-033f6bca8f92 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 06:07:07.338039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:07.338039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 06:07:07.338039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 06:07:07.338039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 06:07:07.338039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 06:07:07.338039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 06:07:07.338039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:07:07.338039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 06:07:07.338039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:07.338039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 06:07:07.338039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:07.338039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:07.338039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:07.338039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:07.338039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:07.338039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:07.339101 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:07.339101 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:07.339101 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:07.339101 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:07.339101 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:07.339101 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:07.339101 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 06:07:07.339101 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 06:07:07.339101 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 06:07:07.339101 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 06:07:07.339101 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:07.339101 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:07.339101 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:07.339101 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:07.339101 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:07.339101 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:07.339956 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:07.339956 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:07.339956 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:07.339956 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:07.339956 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:07.339956 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:07.339956 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:07.339956 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:07.339956 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:07.339956 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:07.339956 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 06:07:07.339956 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:07:07.339956 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 06:07:07.339956 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:07.344523 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 447/870] 10.222.0.40 () {56 vars in 1126 bytes} [Mon Jul 18 06:07:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:07.346343 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7067f0-fd31-4bf6-8c65-3bb060f2676e None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:07.346638 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7067f0-fd31-4bf6-8c65-3bb060f2676e None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:07.346923 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7067f0-fd31-4bf6-8c65-3bb060f2676e None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:07.352374 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 448/871] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:07:07.352374 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7067f0-fd31-4bf6-8c65-3bb060f2676e None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:07.382140 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 424/872] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:07:06 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 06:07:07.384015 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6699f1-5491-4d96-bab6-ce73bbe05a91 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:07.384311 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6699f1-5491-4d96-bab6-ce73bbe05a91 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:07.384651 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6699f1-5491-4d96-bab6-ce73bbe05a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:07.402941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-6f6699f1-5491-4d96-bab6-ce73bbe05a91 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:07.404173 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-6f6699f1-5491-4d96-bab6-ce73bbe05a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:07.488686 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 425/873] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:07 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:07:07.488686 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6699f1-5491-4d96-bab6-ce73bbe05a91 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:07.488950 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7067f0-fd31-4bf6-8c65-3bb060f2676e None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:07.531183 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e81451-a72b-46f6-b455-69a0018fd0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:07.531760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e81451-a72b-46f6-b455-69a0018fd0f2 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:07.532052 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e81451-a72b-46f6-b455-69a0018fd0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:07.532338 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e81451-a72b-46f6-b455-69a0018fd0f2 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:07.533109 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e81451-a72b-46f6-b455-69a0018fd0f2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:07.533422 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e81451-a72b-46f6-b455-69a0018fd0f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:07.534894 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e81451-a72b-46f6-b455-69a0018fd0f2 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:07.542626 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 449/874] 10.222.0.40 () {56 vars in 1122 bytes} [Mon Jul 18 06:07:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:07.543481 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e81451-a72b-46f6-b455-69a0018fd0f2 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:07.565918 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:07.584188 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:07.634546 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6699f1-5491-4d96-bab6-ce73bbe05a91 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:07.667869 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e81451-a72b-46f6-b455-69a0018fd0f2 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:07.680916 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fac74f5-2f51-43f9-a278-8bb08db05bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:07.682012 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac74f5-2f51-43f9-a278-8bb08db05bac None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:07.682299 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac74f5-2f51-43f9-a278-8bb08db05bac None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:07.682577 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac74f5-2f51-43f9-a278-8bb08db05bac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:07.696411 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fac74f5-2f51-43f9-a278-8bb08db05bac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:07.697915 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fac74f5-2f51-43f9-a278-8bb08db05bac None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:07.705142 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d19fb29-3ab5-4f87-8e86-d4e7b6a11f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:07.705970 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19fb29-3ab5-4f87-8e86-d4e7b6a11f04 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:07.706233 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19fb29-3ab5-4f87-8e86-d4e7b6a11f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:07.706508 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19fb29-3ab5-4f87-8e86-d4e7b6a11f04 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:07.707165 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d19fb29-3ab5-4f87-8e86-d4e7b6a11f04 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:07.708571 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d19fb29-3ab5-4f87-8e86-d4e7b6a11f04 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:07.716405 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-9fac74f5-2f51-43f9-a278-8bb08db05bac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:07.716405 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:07.716405 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:07.716405 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:07.716405 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:07.716405 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:07.716405 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:07.716405 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:07.716405 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:07.716405 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:07.716405 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:07.716405 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:07.716405 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:07:07.716405 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:07:07.716405 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:07:07.717612 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:07:07.717612 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:07.717612 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:07.717612 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:07.717612 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:07.717612 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:07.717612 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:07.717612 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:07.717612 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:07.717612 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:07.717612 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:07.717612 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:07.717612 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:07.717612 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:07.717612 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:07.717612 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:07.717612 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:07.718509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:07:07.718509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:07:07.718509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:07.718509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:07.718509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 426/875] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:07:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:07.724104 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fac74f5-2f51-43f9-a278-8bb08db05bac None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:07.724194 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-0d19fb29-3ab5-4f87-8e86-d4e7b6a11f04 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 06:07:07.724194 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:07.724194 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 06:07:07.724194 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 06:07:07.724194 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 06:07:07.724194 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 06:07:07.724194 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 06:07:07.724194 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:07:07.724194 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 06:07:07.724194 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:07.724194 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 06:07:07.724194 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:07.724194 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:07.724194 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:07.724194 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:07.724194 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:07.724194 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:07.725635 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:07.725635 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:07.725635 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:07.725635 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:07.725635 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:07.725635 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:07.725635 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 06:07:07.725635 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 06:07:07.725635 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 06:07:07.725635 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 06:07:07.725635 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:07.725635 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:07.725635 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:07.725635 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:07.725635 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:07.725635 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:07.727612 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:07.727612 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:07.727612 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:07.727612 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:07.727612 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:07.727612 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:07.727612 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:07.727612 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:07.727612 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:07.727612 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:07.727612 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 06:07:07.727612 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:07:07.727612 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 06:07:07.727612 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:07.729442 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 450/876] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 18 06:07:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:07.730019 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1b4fa-a290-4042-bc96-50ab7d91c8f7 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:07.730467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1b4fa-a290-4042-bc96-50ab7d91c8f7 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:07.731337 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1b4fa-a290-4042-bc96-50ab7d91c8f7 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:07.732941 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 451/877] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:07:07.733491 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1b4fa-a290-4042-bc96-50ab7d91c8f7 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:07.834352 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:07.840981 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fac74f5-2f51-43f9-a278-8bb08db05bac None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:07.855252 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1b4fa-a290-4042-bc96-50ab7d91c8f7 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:07.859112 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:07.877592 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4daa9-6b35-48ac-850d-e2e107a0bb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:07.878259 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4daa9-6b35-48ac-850d-e2e107a0bb89 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:07.878548 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4daa9-6b35-48ac-850d-e2e107a0bb89 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:07.878827 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4daa9-6b35-48ac-850d-e2e107a0bb89 None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:07.879543 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a4daa9-6b35-48ac-850d-e2e107a0bb89 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:07.879852 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a4daa9-6b35-48ac-850d-e2e107a0bb89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:07.881263 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a4daa9-6b35-48ac-850d-e2e107a0bb89 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:07.891427 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 427/878] 10.222.0.40 () {56 vars in 1122 bytes} [Mon Jul 18 06:07:07 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:07.891427 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4daa9-6b35-48ac-850d-e2e107a0bb89 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:07.894654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6bafa-1044-4c5f-bbe0-d043f4357467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:07.895355 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6bafa-1044-4c5f-bbe0-d043f4357467 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:07.895750 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6bafa-1044-4c5f-bbe0-d043f4357467 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:07.895924 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6bafa-1044-4c5f-bbe0-d043f4357467 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:07.896720 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb6bafa-1044-4c5f-bbe0-d043f4357467 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:07.897033 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb6bafa-1044-4c5f-bbe0-d043f4357467 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:07.898527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb6bafa-1044-4c5f-bbe0-d043f4357467 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:07.917592 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 452/879] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:07:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:07.917592 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f338d-622b-4842-9064-379988adacd3 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:07.917592 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f338d-622b-4842-9064-379988adacd3 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:07.917592 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f338d-622b-4842-9064-379988adacd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:07.933939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-6f7f338d-622b-4842-9064-379988adacd3 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:07.935168 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-6f7f338d-622b-4842-9064-379988adacd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:08.018661 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 453/880] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:07 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:07:08.019748 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f338d-622b-4842-9064-379988adacd3 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:08.020016 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4daa9-6b35-48ac-850d-e2e107a0bb89 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:08.075892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b9ba9-371e-4202-b2bb-8fcb8958e165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:08.076598 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b9ba9-371e-4202-b2bb-8fcb8958e165 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:08.076881 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b9ba9-371e-4202-b2bb-8fcb8958e165 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:08.077157 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b9ba9-371e-4202-b2bb-8fcb8958e165 None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:08.077870 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b9ba9-371e-4202-b2bb-8fcb8958e165 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:08.078176 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b9ba9-371e-4202-b2bb-8fcb8958e165 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:08.079542 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b9ba9-371e-4202-b2bb-8fcb8958e165 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:08.088554 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 428/881] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 18 06:07:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:08.088554 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b9ba9-371e-4202-b2bb-8fcb8958e165 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:08.123092 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:08.141764 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:08.156806 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f338d-622b-4842-9064-379988adacd3 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:08.202122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c649d-eef3-416e-bdc2-58862235834c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:08.202936 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c649d-eef3-416e-bdc2-58862235834c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:08.203220 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c649d-eef3-416e-bdc2-58862235834c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:08.203495 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c649d-eef3-416e-bdc2-58862235834c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:08.211418 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b9ba9-371e-4202-b2bb-8fcb8958e165 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:08.212222 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c649d-eef3-416e-bdc2-58862235834c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:08.213727 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c649d-eef3-416e-bdc2-58862235834c None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:08.233214 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-454c649d-eef3-416e-bdc2-58862235834c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:07:08.233214 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:08.233214 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:08.233214 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:08.233214 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:08.233214 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:08.233214 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:08.233214 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:08.233214 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:08.233214 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:08.233214 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:08.233214 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:08.233214 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 06:07:08.233214 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 06:07:08.233214 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 06:07:08.235192 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 06:07:08.235192 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:08.235192 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:08.235192 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:08.235192 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:08.235192 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:08.235192 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:08.235192 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:08.235192 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:08.235192 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:08.235192 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:08.235192 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:08.235192 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:08.235192 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:08.235192 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:08.235192 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:08.235192 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:08.237036 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 06:07:08.237036 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 06:07:08.237036 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 06:07:08.237036 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 06:07:08.237036 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 06:07:08.237036 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 06:07:08.237036 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:07:08.237036 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:08.237036 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 454/882] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:07:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:08.237998 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c649d-eef3-416e-bdc2-58862235834c None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:08.283146 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954be34-d441-4d08-b589-c14c075028d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:08.284138 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954be34-d441-4d08-b589-c14c075028d7 None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:08.284719 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954be34-d441-4d08-b589-c14c075028d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:08.285114 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954be34-d441-4d08-b589-c14c075028d7 None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:08.286183 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b954be34-d441-4d08-b589-c14c075028d7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:08.287683 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b954be34-d441-4d08-b589-c14c075028d7 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:08.378794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c649d-eef3-416e-bdc2-58862235834c None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:08.395276 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 429/883] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Jul 18 06:07:08 2022] POST /identity/v3/services => generated 269 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:07:08.396205 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954be34-d441-4d08-b589-c14c075028d7 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:08.430663 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:08.450801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa347673-1457-4bd7-a45e-b90e9afe3502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:08.452230 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa347673-1457-4bd7-a45e-b90e9afe3502 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:08.453901 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa347673-1457-4bd7-a45e-b90e9afe3502 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:08.454510 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa347673-1457-4bd7-a45e-b90e9afe3502 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:08.455733 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa347673-1457-4bd7-a45e-b90e9afe3502 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:08.456977 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa347673-1457-4bd7-a45e-b90e9afe3502 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:08.459203 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa347673-1457-4bd7-a45e-b90e9afe3502 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:08.473850 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 455/884] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 18 06:07:08 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:08.476230 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa347673-1457-4bd7-a45e-b90e9afe3502 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:08.543218 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954be34-d441-4d08-b589-c14c075028d7 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:08.547487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:08.595236 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f038b-b056-449c-af26-6a3671dd1720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:08.596165 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f038b-b056-449c-af26-6a3671dd1720 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:08.596468 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f038b-b056-449c-af26-6a3671dd1720 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:08.596745 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f038b-b056-449c-af26-6a3671dd1720 None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:08.597546 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920f038b-b056-449c-af26-6a3671dd1720 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:08.597856 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920f038b-b056-449c-af26-6a3671dd1720 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:08.599410 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920f038b-b056-449c-af26-6a3671dd1720 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:08.614417 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 430/885] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 18 06:07:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:08.621253 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f038b-b056-449c-af26-6a3671dd1720 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:08.625063 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa347673-1457-4bd7-a45e-b90e9afe3502 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:08.664683 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94999636-0a9f-4c58-8d20-eb7b585a172a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:08.665475 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94999636-0a9f-4c58-8d20-eb7b585a172a None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:08.665861 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94999636-0a9f-4c58-8d20-eb7b585a172a None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:08.666272 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94999636-0a9f-4c58-8d20-eb7b585a172a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:08.667516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94999636-0a9f-4c58-8d20-eb7b585a172a None admin] RBAC: Adding query filter params (interface=public, service_id=d053d9ca7c744f79b9421d6bb0add06b, region_id=RegionOne) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:08.667934 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94999636-0a9f-4c58-8d20-eb7b585a172a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:08.669454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94999636-0a9f-4c58-8d20-eb7b585a172a None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:08.680597 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 456/886] 10.222.0.40 () {56 vars in 1251 bytes} [Mon Jul 18 06:07:08 2022] GET /identity/v3/endpoints?service_id=d053d9ca7c744f79b9421d6bb0add06b&interface=public®ion_id=RegionOne => generated 194 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:08.682185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94999636-0a9f-4c58-8d20-eb7b585a172a None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:08.740370 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f038b-b056-449c-af26-6a3671dd1720 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:08.745754 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:08.780502 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df77c45b-33da-4b8b-87de-3e9a25aae16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:08.781522 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77c45b-33da-4b8b-87de-3e9a25aae16f None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:08.781859 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77c45b-33da-4b8b-87de-3e9a25aae16f None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:08.782148 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77c45b-33da-4b8b-87de-3e9a25aae16f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:08.783937 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df77c45b-33da-4b8b-87de-3e9a25aae16f None admin] RBAC: Adding query filter params (role.id=ab10744ff77646eab23a769ab89c74aa, scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a, user.id=b84dde57ff8e47419866377a819bea21) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:08.784252 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df77c45b-33da-4b8b-87de-3e9a25aae16f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:08.785787 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df77c45b-33da-4b8b-87de-3e9a25aae16f None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:08.793919 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 431/887] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:07:08 2022] GET /identity/v3/role_assignments?user.id=b84dde57ff8e47419866377a819bea21&scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a&role.id=ab10744ff77646eab23a769ab89c74aa => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:07:08.803318 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94999636-0a9f-4c58-8d20-eb7b585a172a None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:08.846429 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402cb67-24c0-460c-96d9-491f1e8c886b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:08.847127 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402cb67-24c0-460c-96d9-491f1e8c886b None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:08.847418 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402cb67-24c0-460c-96d9-491f1e8c886b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:08.847699 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402cb67-24c0-460c-96d9-491f1e8c886b None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:08.848494 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0402cb67-24c0-460c-96d9-491f1e8c886b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:08.853947 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0402cb67-24c0-460c-96d9-491f1e8c886b None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:08.857480 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 457/888] 10.222.0.40 () {56 vars in 1087 bytes} [Mon Jul 18 06:07:08 2022] GET /identity/v3/services => generated 1268 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 06:07:10.051428 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9313d0f-377f-42fd-b239-31799e160e69 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:10.052114 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9313d0f-377f-42fd-b239-31799e160e69 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:10.052114 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9313d0f-377f-42fd-b239-31799e160e69 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:10.053558 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 432/889] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:07:10.059134 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a7865-d270-40bb-80f8-db12b461cf24 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:10.059680 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a7865-d270-40bb-80f8-db12b461cf24 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:10.059680 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a7865-d270-40bb-80f8-db12b461cf24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:10.078278 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-560a7865-d270-40bb-80f8-db12b461cf24 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:10.079511 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-560a7865-d270-40bb-80f8-db12b461cf24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:10.160550 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 458/890] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:10 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:07:10.181869 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9313d0f-377f-42fd-b239-31799e160e69 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:10.210999 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:10.293888 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9313d0f-377f-42fd-b239-31799e160e69 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:10.330246 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d824fa6-5838-41a5-8553-08872a308ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:10.330838 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d824fa6-5838-41a5-8553-08872a308ebb None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:10.331122 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d824fa6-5838-41a5-8553-08872a308ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:10.331403 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d824fa6-5838-41a5-8553-08872a308ebb None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:10.332147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d824fa6-5838-41a5-8553-08872a308ebb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:10.333583 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d824fa6-5838-41a5-8553-08872a308ebb None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:10.413038 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 433/891] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Jul 18 06:07:10 2022] POST /identity/v3/services => generated 241 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:07:10.559000 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1a59f-51fb-4b9d-8dfe-4bcc497636b7 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:10.559000 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1a59f-51fb-4b9d-8dfe-4bcc497636b7 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:10.559000 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1a59f-51fb-4b9d-8dfe-4bcc497636b7 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:10.559790 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 459/892] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:07:10.565886 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66c3a8-4a5c-4499-b19f-0a903a59fd7d None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:10.566303 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66c3a8-4a5c-4499-b19f-0a903a59fd7d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:10.566645 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66c3a8-4a5c-4499-b19f-0a903a59fd7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:10.584789 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-9e66c3a8-4a5c-4499-b19f-0a903a59fd7d None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:10.586445 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-9e66c3a8-4a5c-4499-b19f-0a903a59fd7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:10.670836 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 434/893] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:07:10.696528 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1a59f-51fb-4b9d-8dfe-4bcc497636b7 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:10.751044 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:10.817458 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1a59f-51fb-4b9d-8dfe-4bcc497636b7 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:10.861753 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e6f38-cc35-4842-8cb0-a46e4c3a16c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:10.862715 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e6f38-cc35-4842-8cb0-a46e4c3a16c1 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:10.863001 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e6f38-cc35-4842-8cb0-a46e4c3a16c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:10.863277 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e6f38-cc35-4842-8cb0-a46e4c3a16c1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:10.863952 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e6f38-cc35-4842-8cb0-a46e4c3a16c1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:10.865383 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e6f38-cc35-4842-8cb0-a46e4c3a16c1 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:10.879813 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-8e9e6f38-cc35-4842-8cb0-a46e4c3a16c1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 06:07:10.879813 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:10.879813 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 06:07:10.879813 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 06:07:10.879813 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 06:07:10.879813 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 06:07:10.879813 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 06:07:10.879813 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:07:10.879813 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 06:07:10.879813 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:10.879813 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 06:07:10.879813 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:10.879813 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:10.879813 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:10.879813 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:10.879813 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:10.880894 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:10.880894 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:10.880894 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:10.880894 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:10.880894 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:10.880894 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:10.880894 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:10.880894 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 06:07:10.880894 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 06:07:10.880894 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 06:07:10.880894 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 06:07:10.880894 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:10.880894 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:10.880894 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:10.880894 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:10.880894 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:10.881775 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:10.881775 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:10.881775 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:10.881775 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:10.881775 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:10.881775 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:10.881775 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:10.881775 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:10.881775 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:10.881775 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:10.881775 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:10.881775 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 06:07:10.881775 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:07:10.881775 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 06:07:10.881775 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:10.881775 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 460/894] 10.222.0.40 () {56 vars in 1132 bytes} [Mon Jul 18 06:07:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:10.889533 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e66c3a8-4a5c-4499-b19f-0a903a59fd7d None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:10.909728 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:11.012550 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e66c3a8-4a5c-4499-b19f-0a903a59fd7d None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:11.028669 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b29ab7-cb96-4d63-8382-8b61d07d9de2 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:11.029040 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b29ab7-cb96-4d63-8382-8b61d07d9de2 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:11.029309 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b29ab7-cb96-4d63-8382-8b61d07d9de2 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:11.031077 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 461/895] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:07:11.037134 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94876b-40e6-4032-ab08-0d5229eb4e11 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:11.037422 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94876b-40e6-4032-ab08-0d5229eb4e11 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:11.037701 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94876b-40e6-4032-ab08-0d5229eb4e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:11.054429 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be58ec3-ceb3-4dc1-a3e6-52afe09b36e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:11.055672 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be58ec3-ceb3-4dc1-a3e6-52afe09b36e5 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:11.055917 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be58ec3-ceb3-4dc1-a3e6-52afe09b36e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:11.056218 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be58ec3-ceb3-4dc1-a3e6-52afe09b36e5 None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:11.057057 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be58ec3-ceb3-4dc1-a3e6-52afe09b36e5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:11.057370 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be58ec3-ceb3-4dc1-a3e6-52afe09b36e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:11.058761 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be58ec3-ceb3-4dc1-a3e6-52afe09b36e5 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:11.060630 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-bf94876b-40e6-4032-ab08-0d5229eb4e11 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:11.062432 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-bf94876b-40e6-4032-ab08-0d5229eb4e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:11.064264 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 435/896] 10.222.0.40 () {56 vars in 1126 bytes} [Mon Jul 18 06:07:10 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:11.065631 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b968c-5a5b-422d-a4b0-1549b0d04d8a None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:11.065916 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b968c-5a5b-422d-a4b0-1549b0d04d8a None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:11.066229 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b968c-5a5b-422d-a4b0-1549b0d04d8a None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:11.067676 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 436/897] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:07:11.073280 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990b968c-5a5b-422d-a4b0-1549b0d04d8a None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:11.147873 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 462/898] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 06:07:11.149811 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dfcd99-a4c4-407d-b161-1e105331bb99 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:11.150089 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dfcd99-a4c4-407d-b161-1e105331bb99 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:11.150373 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dfcd99-a4c4-407d-b161-1e105331bb99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:11.171400 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:11.179954 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-93dfcd99-a4c4-407d-b161-1e105331bb99 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:11.181202 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-93dfcd99-a4c4-407d-b161-1e105331bb99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:11.194920 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990b968c-5a5b-422d-a4b0-1549b0d04d8a None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:11.230570 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:11.242125 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c64fb-9e71-4cf8-ad8b-3104a758e97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:11.242805 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c64fb-9e71-4cf8-ad8b-3104a758e97d None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:11.243089 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c64fb-9e71-4cf8-ad8b-3104a758e97d None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:11.243360 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c64fb-9e71-4cf8-ad8b-3104a758e97d None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:11.244075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63c64fb-9e71-4cf8-ad8b-3104a758e97d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:11.244370 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63c64fb-9e71-4cf8-ad8b-3104a758e97d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:11.245767 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63c64fb-9e71-4cf8-ad8b-3104a758e97d None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:11.251313 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 437/899] 10.222.0.40 () {56 vars in 1126 bytes} [Mon Jul 18 06:07:11 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:11.252115 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c64fb-9e71-4cf8-ad8b-3104a758e97d None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:11.282245 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 463/900] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:07:11.283431 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dfcd99-a4c4-407d-b161-1e105331bb99 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:11.405022 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c64fb-9e71-4cf8-ad8b-3104a758e97d None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:11.409931 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dfcd99-a4c4-407d-b161-1e105331bb99 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:11.448258 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf14bf4-4ea7-41a0-866d-11c8b3b3754c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:11.449087 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf14bf4-4ea7-41a0-866d-11c8b3b3754c None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:11.449367 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf14bf4-4ea7-41a0-866d-11c8b3b3754c None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:11.449638 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf14bf4-4ea7-41a0-866d-11c8b3b3754c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:11.453157 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f838a6-40f9-4487-91dd-9d71cc4431ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:11.453828 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f838a6-40f9-4487-91dd-9d71cc4431ba None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:11.454119 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f838a6-40f9-4487-91dd-9d71cc4431ba None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:11.454463 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f838a6-40f9-4487-91dd-9d71cc4431ba None admin] PATH_INFO: `/v3/endpoints` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:11.455367 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f838a6-40f9-4487-91dd-9d71cc4431ba None admin] RBAC: Adding query filter params (interface=public, service_id=877ab5fa6ded461e9dd5f1214f7b7593, region_id=RegionOne) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:11.455730 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f838a6-40f9-4487-91dd-9d71cc4431ba None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:11.457305 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f838a6-40f9-4487-91dd-9d71cc4431ba None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:11.461938 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf14bf4-4ea7-41a0-866d-11c8b3b3754c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:07:11.466322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 464/901] 10.222.0.40 () {56 vars in 1251 bytes} [Mon Jul 18 06:07:11 2022] GET /identity/v3/endpoints?service_id=877ab5fa6ded461e9dd5f1214f7b7593&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:11.467332 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f838a6-40f9-4487-91dd-9d71cc4431ba None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:11.470444 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-fcf14bf4-4ea7-41a0-866d-11c8b3b3754c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 06:07:11.470444 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:11.470444 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:11.470444 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:11.470444 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:11.470444 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:11.470444 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:11.470444 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:11.470444 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:11.470444 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:11.470444 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:11.470444 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:11.470444 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:07:11.470444 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:07:11.470444 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:07:11.470444 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:07:11.471387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:07:11.471387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:07:11.471387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:07:11.471387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:07:11.471387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:11.471387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:11.471387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:11.471387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:11.471387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:11.471387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:11.471387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:11.471387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:11.471387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:11.471387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:11.471387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:11.471387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:11.472252 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:11.472252 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:11.472252 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:11.472252 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:11.472252 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:07:11.472252 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:07:11.472252 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:07:11.472252 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:07:11.472252 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:07:11.472252 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:07:11.472252 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 06:07:11.472252 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:11.476707 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 438/902] 10.222.0.40 () {56 vars in 1120 bytes} [Mon Jul 18 06:07:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:11.477448 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf14bf4-4ea7-41a0-866d-11c8b3b3754c None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:11.480216 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:11.511819 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:11.580701 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f838a6-40f9-4487-91dd-9d71cc4431ba None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:11.584665 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf14bf4-4ea7-41a0-866d-11c8b3b3754c None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:11.618573 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f59872-2e6a-4a38-b4d3-6faee4e605bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:11.619410 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f59872-2e6a-4a38-b4d3-6faee4e605bb None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:11.619699 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f59872-2e6a-4a38-b4d3-6faee4e605bb None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:11.619977 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f59872-2e6a-4a38-b4d3-6faee4e605bb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:11.620693 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f59872-2e6a-4a38-b4d3-6faee4e605bb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:11.622247 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f59872-2e6a-4a38-b4d3-6faee4e605bb None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:11.623198 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91509c0e-1488-40bf-bfee-ddd933f4701b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:11.624046 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91509c0e-1488-40bf-bfee-ddd933f4701b None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:11.624519 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91509c0e-1488-40bf-bfee-ddd933f4701b None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:11.624941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91509c0e-1488-40bf-bfee-ddd933f4701b None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:11.625778 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91509c0e-1488-40bf-bfee-ddd933f4701b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:11.627396 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91509c0e-1488-40bf-bfee-ddd933f4701b None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:11.634635 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 439/903] 10.222.0.40 () {56 vars in 1087 bytes} [Mon Jul 18 06:07:11 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:07:11.635671 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91509c0e-1488-40bf-bfee-ddd933f4701b None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:11.635779 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-f5f59872-2e6a-4a38-b4d3-6faee4e605bb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 06:07:11.635779 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:11.635779 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 06:07:11.635779 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 06:07:11.635779 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 06:07:11.635779 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 06:07:11.635779 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 06:07:11.635779 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:07:11.635779 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 06:07:11.635779 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:11.635779 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 06:07:11.635779 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:11.635779 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:11.635779 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:11.635779 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:11.635779 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:11.635779 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:11.636751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:11.636751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:11.636751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:11.636751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:11.636751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:11.636751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:11.636751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 06:07:11.636751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 06:07:11.636751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 06:07:11.636751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 06:07:11.636751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:11.636751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:11.636751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:11.636751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:11.636751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:11.636751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:11.637572 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:11.637572 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:11.637572 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:11.637572 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:11.637572 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:11.637572 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:11.637572 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:11.637572 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:11.637572 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:11.637572 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:11.637572 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 06:07:11.637572 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:07:11.637572 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 06:07:11.637572 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:11.637572 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 465/904] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 18 06:07:11 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:11.646772 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f59872-2e6a-4a38-b4d3-6faee4e605bb None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:11.728579 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:11.769526 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:11.772789 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f59872-2e6a-4a38-b4d3-6faee4e605bb None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:11.772965 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91509c0e-1488-40bf-bfee-ddd933f4701b None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:11.813413 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445be6a8-10af-48ca-bb66-48247019ff08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:11.814545 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445be6a8-10af-48ca-bb66-48247019ff08 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:11.814980 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445be6a8-10af-48ca-bb66-48247019ff08 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:11.815429 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445be6a8-10af-48ca-bb66-48247019ff08 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:11.815541 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbde2d6-c634-45be-9018-d843b985a919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:11.816154 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbde2d6-c634-45be-9018-d843b985a919 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:11.816519 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbde2d6-c634-45be-9018-d843b985a919 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:11.816746 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445be6a8-10af-48ca-bb66-48247019ff08 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:11.816868 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbde2d6-c634-45be-9018-d843b985a919 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:11.817296 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445be6a8-10af-48ca-bb66-48247019ff08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:11.817552 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbde2d6-c634-45be-9018-d843b985a919 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:11.817943 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbde2d6-c634-45be-9018-d843b985a919 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:11.819170 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445be6a8-10af-48ca-bb66-48247019ff08 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:11.819401 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbde2d6-c634-45be-9018-d843b985a919 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:11.830454 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 440/906] 10.222.0.40 () {56 vars in 1115 bytes} [Mon Jul 18 06:07:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:11.830795 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 466/906] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 18 06:07:11 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:11.831306 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbde2d6-c634-45be-9018-d843b985a919 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:11.838313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445be6a8-10af-48ca-bb66-48247019ff08 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:11.954046 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445be6a8-10af-48ca-bb66-48247019ff08 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:11.960444 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbde2d6-c634-45be-9018-d843b985a919 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:11.992395 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:12.001074 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a3ab3-a678-4612-91f4-3443fbd48af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:12.001816 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a3ab3-a678-4612-91f4-3443fbd48af6 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:12.002104 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a3ab3-a678-4612-91f4-3443fbd48af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:12.002387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a3ab3-a678-4612-91f4-3443fbd48af6 None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:12.003109 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10a3ab3-a678-4612-91f4-3443fbd48af6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:12.003416 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10a3ab3-a678-4612-91f4-3443fbd48af6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:12.004608 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e4a4c0-7b6f-4ef7-90c0-af6094030efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:12.004882 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10a3ab3-a678-4612-91f4-3443fbd48af6 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:12.005414 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4a4c0-7b6f-4ef7-90c0-af6094030efe None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:12.005701 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4a4c0-7b6f-4ef7-90c0-af6094030efe None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:12.005975 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4a4c0-7b6f-4ef7-90c0-af6094030efe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:12.010553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 441/907] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 18 06:07:11 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:07:12.017824 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a3ab3-a678-4612-91f4-3443fbd48af6 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:12.020216 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e4a4c0-7b6f-4ef7-90c0-af6094030efe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:12.020647 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e4a4c0-7b6f-4ef7-90c0-af6094030efe None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:12.047576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-c8e4a4c0-7b6f-4ef7-90c0-af6094030efe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:12.047576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:12.047576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:12.047576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:12.047576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:12.047576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:12.047576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:12.047576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:12.047576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:12.047576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:12.047576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:12.047576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:12.047576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:07:12.047576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:07:12.047576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:07:12.048777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:07:12.048777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:12.048777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:12.048777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:12.048777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:12.048777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:12.048777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:12.048777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:12.048777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:12.048777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:12.048777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:12.048777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:12.048777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:12.048777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:12.048777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:12.048777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:12.048777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:12.049710 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:07:12.049710 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:07:12.049710 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:12.049710 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:12.052495 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 467/908] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:07:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:07:12.055902 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e4a4c0-7b6f-4ef7-90c0-af6094030efe None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:12.081231 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:12.133405 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a3ab3-a678-4612-91f4-3443fbd48af6 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:12.179182 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee9e47-76d5-45ce-ab26-3e7e63f75d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:12.179903 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee9e47-76d5-45ce-ab26-3e7e63f75d0c None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:12.180458 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee9e47-76d5-45ce-ab26-3e7e63f75d0c None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:12.180750 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee9e47-76d5-45ce-ab26-3e7e63f75d0c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:12.181923 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cee9e47-76d5-45ce-ab26-3e7e63f75d0c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:12.183567 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cee9e47-76d5-45ce-ab26-3e7e63f75d0c None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:12.188055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e4a4c0-7b6f-4ef7-90c0-af6094030efe None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:12.227707 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7b52a-177d-4b34-ad9c-4256c96b8c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:12.228307 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7b52a-177d-4b34-ad9c-4256c96b8c68 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:12.228705 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7b52a-177d-4b34-ad9c-4256c96b8c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:12.228988 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7b52a-177d-4b34-ad9c-4256c96b8c68 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:12.229729 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7b52a-177d-4b34-ad9c-4256c96b8c68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:12.230044 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7b52a-177d-4b34-ad9c-4256c96b8c68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:12.231522 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7b52a-177d-4b34-ad9c-4256c96b8c68 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:12.244571 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 468/909] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:07:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:12.250208 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7b52a-177d-4b34-ad9c-4256c96b8c68 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:12.272462 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 442/910] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Jul 18 06:07:12 2022] POST /identity/v3/endpoints => generated 346 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:07:12.350216 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:12.378411 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7b52a-177d-4b34-ad9c-4256c96b8c68 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:12.428107 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac336ea-d336-4de0-9386-08ce40e092a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:12.428964 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac336ea-d336-4de0-9386-08ce40e092a3 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:12.429262 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac336ea-d336-4de0-9386-08ce40e092a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:12.429582 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac336ea-d336-4de0-9386-08ce40e092a3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:12.438090 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac336ea-d336-4de0-9386-08ce40e092a3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:12.439549 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac336ea-d336-4de0-9386-08ce40e092a3 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:12.455191 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-8ac336ea-d336-4de0-9386-08ce40e092a3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 06:07:12.455191 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:12.455191 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:12.455191 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:12.455191 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:12.455191 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:12.455191 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:12.455191 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:12.455191 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:12.455191 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:12.455191 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:12.455191 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:12.455191 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 06:07:12.455191 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 06:07:12.455191 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 06:07:12.455191 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 06:07:12.456216 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:12.456216 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:12.456216 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 06:07:12.456216 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:07:12.456216 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 06:07:12.456216 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:07:12.456216 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:12.456216 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:12.456216 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:12.456216 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:12.456216 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:12.456216 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:12.456216 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:12.456216 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:12.456216 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:12.456216 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:12.456216 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:12.456216 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:12.457292 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:12.457292 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:12.457292 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 06:07:12.457292 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 06:07:12.457292 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 06:07:12.457292 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 06:07:12.457292 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 06:07:12.457292 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 06:07:12.457292 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 06:07:12.457292 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:12.457292 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 469/911] 10.222.0.40 () {56 vars in 1117 bytes} [Mon Jul 18 06:07:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:07:12.464984 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee9e47-76d5-45ce-ab26-3e7e63f75d0c None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:12.549193 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:12.552529 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5a4f2-811c-4c92-8048-82b36a8e1bb8 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:12.552819 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5a4f2-811c-4c92-8048-82b36a8e1bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:12.553100 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5a4f2-811c-4c92-8048-82b36a8e1bb8 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:12.554771 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 470/912] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:07:12.560119 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d37cd-f88e-466e-b049-3a5654ad3330 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:12.565224 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d37cd-f88e-466e-b049-3a5654ad3330 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:12.565224 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d37cd-f88e-466e-b049-3a5654ad3330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:12.579269 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-773d37cd-f88e-466e-b049-3a5654ad3330 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:12.580505 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-773d37cd-f88e-466e-b049-3a5654ad3330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:12.628649 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee9e47-76d5-45ce-ab26-3e7e63f75d0c None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:12.676017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df87940c-d497-426e-b664-bf3347874942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:12.677957 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87940c-d497-426e-b664-bf3347874942 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:12.678464 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87940c-d497-426e-b664-bf3347874942 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:12.678948 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87940c-d497-426e-b664-bf3347874942 None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:12.680558 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df87940c-d497-426e-b664-bf3347874942 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:12.681808 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df87940c-d497-426e-b664-bf3347874942 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:12.684491 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df87940c-d497-426e-b664-bf3347874942 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:12.704702 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 471/913] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 06:07:12.722934 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d37cd-f88e-466e-b049-3a5654ad3330 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:12.728489 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 443/914] 10.222.0.40 () {56 vars in 1149 bytes} [Mon Jul 18 06:07:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:12.736436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df87940c-d497-426e-b664-bf3347874942 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:12.792994 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:12.832873 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d37cd-f88e-466e-b049-3a5654ad3330 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:12.873893 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df87940c-d497-426e-b664-bf3347874942 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:12.881530 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9ee14-a5ca-4dbb-a159-23f8689d037e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:12.883289 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:12.884152 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9ee14-a5ca-4dbb-a159-23f8689d037e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:12.884600 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9ee14-a5ca-4dbb-a159-23f8689d037e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:12.884890 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9ee14-a5ca-4dbb-a159-23f8689d037e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:12.885617 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a9ee14-a5ca-4dbb-a159-23f8689d037e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:12.887428 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a9ee14-a5ca-4dbb-a159-23f8689d037e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:12.904234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-49a9ee14-a5ca-4dbb-a159-23f8689d037e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 06:07:12.904234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:12.904234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 06:07:12.904234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 06:07:12.904234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 06:07:12.904234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 06:07:12.904234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 06:07:12.904234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:07:12.904234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 06:07:12.904234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:12.904234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 06:07:12.904234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:12.904234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:12.904234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:12.904234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:12.904234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:12.904234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:12.906178 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:12.906178 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:12.906178 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:12.906178 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:12.906178 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:12.906178 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:12.906178 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 06:07:12.906178 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 06:07:12.906178 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 06:07:12.906178 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 06:07:12.906178 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:12.906178 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:12.906178 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:12.906178 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:12.906178 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:12.906178 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:12.907352 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:12.907352 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:12.907352 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:12.907352 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:12.907352 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:12.907352 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:12.907352 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:12.907352 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:12.907352 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:12.907352 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:12.907352 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 06:07:12.907352 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:07:12.907352 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 06:07:12.907352 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:12.907352 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 472/915] 10.222.0.40 () {56 vars in 1126 bytes} [Mon Jul 18 06:07:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:12.914369 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9ee14-a5ca-4dbb-a159-23f8689d037e None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:12.922478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a3dfc-8303-4553-a81c-fae7b675dfc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:12.923209 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a3dfc-8303-4553-a81c-fae7b675dfc5 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:12.923567 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a3dfc-8303-4553-a81c-fae7b675dfc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:12.923908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a3dfc-8303-4553-a81c-fae7b675dfc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:13.023431 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2a3dfc-8303-4553-a81c-fae7b675dfc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:13.025891 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2a3dfc-8303-4553-a81c-fae7b675dfc5 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:13.034348 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9ee14-a5ca-4dbb-a159-23f8689d037e None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:13.071614 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:13.080969 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726444f8-16db-41b1-af92-ee59d2ac25fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:13.081836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726444f8-16db-41b1-af92-ee59d2ac25fc None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:13.082404 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726444f8-16db-41b1-af92-ee59d2ac25fc None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:13.082635 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726444f8-16db-41b1-af92-ee59d2ac25fc None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:13.094462 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726444f8-16db-41b1-af92-ee59d2ac25fc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:13.094462 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726444f8-16db-41b1-af92-ee59d2ac25fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:13.094462 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726444f8-16db-41b1-af92-ee59d2ac25fc None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:13.095046 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 473/916] 10.222.0.40 () {56 vars in 1122 bytes} [Mon Jul 18 06:07:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:07:13.096458 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726444f8-16db-41b1-af92-ee59d2ac25fc None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:13.143887 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 444/917] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:07:12 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 06:07:13.154043 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a3dfc-8303-4553-a81c-fae7b675dfc5 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:13.164289 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:13.218414 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726444f8-16db-41b1-af92-ee59d2ac25fc None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:13.255155 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1951d-3b44-4e6e-adab-7f54e143a5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:13.255828 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1951d-3b44-4e6e-adab-7f54e143a5b6 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:13.256212 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1951d-3b44-4e6e-adab-7f54e143a5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:13.256521 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1951d-3b44-4e6e-adab-7f54e143a5b6 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:13.257359 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe1951d-3b44-4e6e-adab-7f54e143a5b6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:13.257702 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe1951d-3b44-4e6e-adab-7f54e143a5b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:13.259085 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe1951d-3b44-4e6e-adab-7f54e143a5b6 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:13.263361 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a3dfc-8303-4553-a81c-fae7b675dfc5 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:13.264763 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 474/918] 10.222.0.40 () {56 vars in 1122 bytes} [Mon Jul 18 06:07:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:13.271000 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1951d-3b44-4e6e-adab-7f54e143a5b6 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:13.311895 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e35175-b7c6-462e-a209-002efad08c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:13.312733 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e35175-b7c6-462e-a209-002efad08c0f None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:13.313020 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e35175-b7c6-462e-a209-002efad08c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:13.313299 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e35175-b7c6-462e-a209-002efad08c0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:13.331386 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e35175-b7c6-462e-a209-002efad08c0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:13.333086 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e35175-b7c6-462e-a209-002efad08c0f None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:13.345982 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:13.356624 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-d0e35175-b7c6-462e-a209-002efad08c0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:13.356624 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:13.356624 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:13.356624 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:13.356624 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:13.356624 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:13.356624 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:13.356624 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:13.356624 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:13.356624 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:13.356624 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:13.356624 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:13.356624 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:07:13.356624 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:07:13.356624 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:07:13.358609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:07:13.358609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:13.358609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:13.358609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:13.358609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:13.358609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:13.358609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:13.358609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:13.358609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:13.358609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:13.358609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:13.358609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:13.358609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:13.358609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:13.358609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:13.358609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:13.358609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:13.359729 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:07:13.359729 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:07:13.359729 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:13.359729 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:13.360559 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 445/919] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:07:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:13.366709 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e35175-b7c6-462e-a209-002efad08c0f None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:13.409659 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1951d-3b44-4e6e-adab-7f54e143a5b6 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:13.443928 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:13.463588 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c73c8-f408-4c9a-bf63-c3f2f7174a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:13.464290 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c73c8-f408-4c9a-bf63-c3f2f7174a73 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:13.464626 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c73c8-f408-4c9a-bf63-c3f2f7174a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:13.464950 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c73c8-f408-4c9a-bf63-c3f2f7174a73 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:13.465723 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927c73c8-f408-4c9a-bf63-c3f2f7174a73 None admin] RBAC: Adding query filter params (interface=public, service_id=9e05e550ea8f4422a52f1c6dfad15750, region_id=RegionOne) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:13.466042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927c73c8-f408-4c9a-bf63-c3f2f7174a73 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:13.467741 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927c73c8-f408-4c9a-bf63-c3f2f7174a73 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:13.480345 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 475/920] 10.222.0.40 () {56 vars in 1251 bytes} [Mon Jul 18 06:07:13 2022] GET /identity/v3/endpoints?service_id=9e05e550ea8f4422a52f1c6dfad15750&interface=public®ion_id=RegionOne => generated 194 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:13.484880 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c73c8-f408-4c9a-bf63-c3f2f7174a73 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:13.513741 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e35175-b7c6-462e-a209-002efad08c0f None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:13.553111 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d56bcbc-caac-4aae-bf55-6216c8d1474d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:13.554201 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56bcbc-caac-4aae-bf55-6216c8d1474d None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:13.555256 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56bcbc-caac-4aae-bf55-6216c8d1474d None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:13.555761 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56bcbc-caac-4aae-bf55-6216c8d1474d None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:13.556894 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d56bcbc-caac-4aae-bf55-6216c8d1474d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:13.557430 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d56bcbc-caac-4aae-bf55-6216c8d1474d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:13.559066 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d56bcbc-caac-4aae-bf55-6216c8d1474d None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:13.575362 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 446/921] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:07:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:07:13.580619 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d56bcbc-caac-4aae-bf55-6216c8d1474d None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:13.633633 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c73c8-f408-4c9a-bf63-c3f2f7174a73 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:13.662027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:13.689880 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c632922c-f45c-4e21-9ac1-1750822aa93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:13.690578 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c632922c-f45c-4e21-9ac1-1750822aa93c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:13.690864 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c632922c-f45c-4e21-9ac1-1750822aa93c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:13.691144 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c632922c-f45c-4e21-9ac1-1750822aa93c None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:13.692258 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c632922c-f45c-4e21-9ac1-1750822aa93c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:13.693682 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c632922c-f45c-4e21-9ac1-1750822aa93c None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:13.708440 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 476/922] 10.222.0.40 () {56 vars in 1087 bytes} [Mon Jul 18 06:07:13 2022] GET /identity/v3/services => generated 1497 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:07:13.709323 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d56bcbc-caac-4aae-bf55-6216c8d1474d None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:13.731695 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:13.751807 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8152dfd-5ccf-4160-9ab5-ae5b543438d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:13.752657 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8152dfd-5ccf-4160-9ab5-ae5b543438d7 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:13.752945 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8152dfd-5ccf-4160-9ab5-ae5b543438d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:13.753225 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8152dfd-5ccf-4160-9ab5-ae5b543438d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:13.761943 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8152dfd-5ccf-4160-9ab5-ae5b543438d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:13.763473 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8152dfd-5ccf-4160-9ab5-ae5b543438d7 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:13.783063 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-c8152dfd-5ccf-4160-9ab5-ae5b543438d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:07:13.783063 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:13.783063 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:13.783063 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:13.783063 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:13.783063 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:13.783063 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:13.783063 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:13.783063 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:13.783063 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:13.783063 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:13.783063 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:13.783063 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 06:07:13.783063 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 06:07:13.783063 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 06:07:13.784035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 06:07:13.784035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:13.784035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:13.784035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:13.784035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:13.784035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:13.784035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:13.784035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:13.784035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:13.784035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:13.784035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:13.784035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:13.784035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:13.784035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:13.784035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:13.784035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:13.784035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:13.790458 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 06:07:13.790458 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 06:07:13.790458 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 06:07:13.790458 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 06:07:13.790458 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 06:07:13.790458 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 06:07:13.790458 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:07:13.790458 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:13.790458 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 447/923] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:07:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:07:13.792397 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c632922c-f45c-4e21-9ac1-1750822aa93c None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:13.927277 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c632922c-f45c-4e21-9ac1-1750822aa93c None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:13.953828 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3885c4a-63fb-4d56-9f41-ddf15b447c83 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:13.954147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3885c4a-63fb-4d56-9f41-ddf15b447c83 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:13.954390 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3885c4a-63fb-4d56-9f41-ddf15b447c83 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:13.956071 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 448/924] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:07:13.961958 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac87b21-3144-46da-99aa-b9669027ab07 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:13.962245 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac87b21-3144-46da-99aa-b9669027ab07 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:13.962505 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac87b21-3144-46da-99aa-b9669027ab07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:13.971130 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5cb170-7165-4c45-a6a7-94f78d6e2c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:13.972199 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5cb170-7165-4c45-a6a7-94f78d6e2c6a None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:13.972797 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5cb170-7165-4c45-a6a7-94f78d6e2c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:13.973272 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5cb170-7165-4c45-a6a7-94f78d6e2c6a None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:13.974434 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5cb170-7165-4c45-a6a7-94f78d6e2c6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:13.974753 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5cb170-7165-4c45-a6a7-94f78d6e2c6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:13.976326 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5cb170-7165-4c45-a6a7-94f78d6e2c6a None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:13.980782 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-5ac87b21-3144-46da-99aa-b9669027ab07 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:13.982282 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-5ac87b21-3144-46da-99aa-b9669027ab07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:13.990804 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 477/925] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 18 06:07:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:13.998527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5cb170-7165-4c45-a6a7-94f78d6e2c6a None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:14.049842 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:14.063074 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 449/926] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 06:07:14.095910 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac87b21-3144-46da-99aa-b9669027ab07 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:14.138776 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5cb170-7165-4c45-a6a7-94f78d6e2c6a None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:14.201145 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf8dbd-24d1-4ad9-9700-65d283aa226e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:14.201887 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf8dbd-24d1-4ad9-9700-65d283aa226e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:14.202177 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf8dbd-24d1-4ad9-9700-65d283aa226e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:14.202461 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf8dbd-24d1-4ad9-9700-65d283aa226e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:14.203319 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bf8dbd-24d1-4ad9-9700-65d283aa226e None admin] RBAC: Adding query filter params (role.id=34f769e98e1e449cb45aa4b31836393a, scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a, user.id=b84dde57ff8e47419866377a819bea21) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:14.203634 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bf8dbd-24d1-4ad9-9700-65d283aa226e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:14.205453 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bf8dbd-24d1-4ad9-9700-65d283aa226e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:14.214401 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 478/927] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:07:13 2022] GET /identity/v3/role_assignments?user.id=b84dde57ff8e47419866377a819bea21&scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a&role.id=34f769e98e1e449cb45aa4b31836393a => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:07:14.235476 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac87b21-3144-46da-99aa-b9669027ab07 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:14.247920 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:14.281182 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b18a3a-e1aa-43ef-8265-b5919ac921f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:14.282029 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b18a3a-e1aa-43ef-8265-b5919ac921f8 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:14.282309 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b18a3a-e1aa-43ef-8265-b5919ac921f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:14.282585 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b18a3a-e1aa-43ef-8265-b5919ac921f8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:14.283271 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b18a3a-e1aa-43ef-8265-b5919ac921f8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:14.284786 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b18a3a-e1aa-43ef-8265-b5919ac921f8 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:14.300135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-47b18a3a-e1aa-43ef-8265-b5919ac921f8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 06:07:14.300135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:14.300135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 06:07:14.300135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 06:07:14.300135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 06:07:14.300135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 06:07:14.300135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 06:07:14.300135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:07:14.300135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 06:07:14.300135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:14.300135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 06:07:14.300135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:14.300135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:14.300135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:14.300135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:14.300135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:14.301806 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:14.301806 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:14.301806 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:14.301806 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:14.301806 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:14.301806 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:14.301806 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:14.301806 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 06:07:14.301806 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 06:07:14.301806 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 06:07:14.301806 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 06:07:14.301806 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:14.301806 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:14.301806 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:14.301806 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:14.301806 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:14.303235 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:14.303235 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:14.303235 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:14.303235 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:14.303235 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:14.303235 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:14.303235 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:14.303235 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:14.303235 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:14.303235 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:14.303235 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:14.303235 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 06:07:14.303235 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:07:14.303235 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 06:07:14.303235 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:14.303235 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 450/928] 10.222.0.40 () {56 vars in 1132 bytes} [Mon Jul 18 06:07:14 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:14.307713 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf8dbd-24d1-4ad9-9700-65d283aa226e None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:14.426793 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511cdc3-9c16-492b-83bf-97bafb0fa40d None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:14.427443 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511cdc3-9c16-492b-83bf-97bafb0fa40d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:14.427945 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511cdc3-9c16-492b-83bf-97bafb0fa40d None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:14.430753 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 451/929] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:07:14.440285 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9910418-970e-4e95-924c-ba3ccc583433 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:14.441185 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9910418-970e-4e95-924c-ba3ccc583433 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:14.441918 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9910418-970e-4e95-924c-ba3ccc583433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:14.447707 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf8dbd-24d1-4ad9-9700-65d283aa226e None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:14.456506 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:14.470766 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-c9910418-970e-4e95-924c-ba3ccc583433 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:14.472017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-c9910418-970e-4e95-924c-ba3ccc583433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:14.503383 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9029f3-9349-4230-adc0-ee81abf475b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:14.505081 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9029f3-9349-4230-adc0-ee81abf475b5 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:14.505585 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9029f3-9349-4230-adc0-ee81abf475b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:14.506049 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9029f3-9349-4230-adc0-ee81abf475b5 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:14.508191 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9029f3-9349-4230-adc0-ee81abf475b5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:14.509365 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9029f3-9349-4230-adc0-ee81abf475b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:14.511073 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9029f3-9349-4230-adc0-ee81abf475b5 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:14.518582 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 479/930] 10.222.0.40 () {56 vars in 1126 bytes} [Mon Jul 18 06:07:14 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:14.526625 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9029f3-9349-4230-adc0-ee81abf475b5 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:14.565581 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 452/931] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:07:14.592578 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9910418-970e-4e95-924c-ba3ccc583433 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:14.678999 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9029f3-9349-4230-adc0-ee81abf475b5 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:14.731449 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca97914a-e386-410f-823c-064ea8670f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:14.733228 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97914a-e386-410f-823c-064ea8670f46 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:14.733797 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97914a-e386-410f-823c-064ea8670f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:14.734222 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97914a-e386-410f-823c-064ea8670f46 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:14.735375 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca97914a-e386-410f-823c-064ea8670f46 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:14.735839 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca97914a-e386-410f-823c-064ea8670f46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:14.738253 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca97914a-e386-410f-823c-064ea8670f46 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:14.740373 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9910418-970e-4e95-924c-ba3ccc583433 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:14.744126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:14.747626 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 480/932] 10.222.0.40 () {56 vars in 1126 bytes} [Mon Jul 18 06:07:14 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:07:14.754288 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca97914a-e386-410f-823c-064ea8670f46 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:14.792640 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c54db-41ce-4ceb-bede-4c5c4cf73d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:14.794102 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c54db-41ce-4ceb-bede-4c5c4cf73d61 None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:14.794645 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c54db-41ce-4ceb-bede-4c5c4cf73d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:14.795360 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c54db-41ce-4ceb-bede-4c5c4cf73d61 None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:14.796690 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c54db-41ce-4ceb-bede-4c5c4cf73d61 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:14.798966 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c54db-41ce-4ceb-bede-4c5c4cf73d61 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:14.800994 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: WARNING keystone.server.flask.common [None req-fd0c54db-41ce-4ceb-bede-4c5c4cf73d61 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 18 06:07:14.812982 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:14.863315 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.sql.core [None req-fd0c54db-41ce-4ceb-bede-4c5c4cf73d61 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 18 06:07:14.863315 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 18 06:07:14.863315 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [parameters: {'id': '0a70a48a87dd46cbb6a3f593325e08f5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 18 06:07:14.863315 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80471) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 18 06:07:14.883430 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-fd0c54db-41ce-4ceb-bede-4c5c4cf73d61 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0a70a48a87dd46cbb6a3f593325e08f5.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0a70a48a87dd46cbb6a3f593325e08f5. Jul 18 06:07:14.883430 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:14.883430 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 06:07:14.883430 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 06:07:14.883430 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 06:07:14.883430 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 06:07:14.883430 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 06:07:14.883430 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 06:07:14.883430 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 06:07:14.883430 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application conn.query(q) Jul 18 06:07:14.883430 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 06:07:14.883430 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 06:07:14.883430 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 06:07:14.884970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application result.read() Jul 18 06:07:14.884970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 06:07:14.884970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 06:07:14.884970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 06:07:14.884970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 06:07:14.884970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 06:07:14.884970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 06:07:14.884970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 06:07:14.884970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 06:07:14.884970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 18 06:07:14.884970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:14.884970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 18 06:07:14.884970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:14.884970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:14.884970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 18 06:07:14.884970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 18 06:07:14.884970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 18 06:07:14.884970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 18 06:07:14.886781 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 06:07:14.886781 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application next(self.gen) Jul 18 06:07:14.886781 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 18 06:07:14.886781 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application yield resource Jul 18 06:07:14.886781 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 06:07:14.886781 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application next(self.gen) Jul 18 06:07:14.886781 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 18 06:07:14.886781 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self.session.rollback() Jul 18 06:07:14.886781 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 18 06:07:14.886781 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self.force_reraise() Jul 18 06:07:14.886781 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 18 06:07:14.886781 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise self.value Jul 18 06:07:14.886781 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 18 06:07:14.886781 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 18 06:07:14.886781 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 18 06:07:14.886781 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application session.commit() Jul 18 06:07:14.886781 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 18 06:07:14.888477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 18 06:07:14.888477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 18 06:07:14.888477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._prepare_impl() Jul 18 06:07:14.888477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 18 06:07:14.888477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self.session.flush() Jul 18 06:07:14.888477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 18 06:07:14.888477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._flush(objects) Jul 18 06:07:14.888477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 18 06:07:14.888477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 18 06:07:14.888477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 18 06:07:14.888477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application compat.raise_( Jul 18 06:07:14.888477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 06:07:14.888477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception Jul 18 06:07:14.888477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 18 06:07:14.888477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application flush_context.execute() Jul 18 06:07:14.888477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 18 06:07:14.888477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rec.execute(self) Jul 18 06:07:14.890219 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 18 06:07:14.890219 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 18 06:07:14.890219 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 18 06:07:14.890219 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 18 06:07:14.890219 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 18 06:07:14.890219 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 18 06:07:14.890219 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 18 06:07:14.890219 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 18 06:07:14.890219 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 18 06:07:14.890219 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 18 06:07:14.890219 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 18 06:07:14.890219 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 18 06:07:14.890219 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 18 06:07:14.890219 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 18 06:07:14.890219 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 18 06:07:14.891353 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 18 06:07:14.891353 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 06:07:14.891353 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception Jul 18 06:07:14.891353 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 06:07:14.891353 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 06:07:14.891353 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 06:07:14.891353 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 06:07:14.891353 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 06:07:14.891353 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 06:07:14.891353 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 06:07:14.891353 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application conn.query(q) Jul 18 06:07:14.891353 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 06:07:14.891353 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 06:07:14.891353 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 06:07:14.891353 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application result.read() Jul 18 06:07:14.891353 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 06:07:14.891353 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 06:07:14.895404 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 06:07:14.895404 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 06:07:14.895404 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 06:07:14.895404 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 06:07:14.895404 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 06:07:14.895404 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 06:07:14.895404 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 18 06:07:14.895404 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 18 06:07:14.895404 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [parameters: {'id': '0a70a48a87dd46cbb6a3f593325e08f5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 18 06:07:14.895404 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 18 06:07:14.895404 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:14.895404 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 06:07:14.895404 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:14.895404 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:14.895404 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 18 06:07:14.900739 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 18 06:07:14.900739 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 18 06:07:14.900739 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 18 06:07:14.900739 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 18 06:07:14.900739 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:14.900739 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 06:07:14.900739 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:14.900739 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:14.900739 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:14.900739 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:14.900739 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:14.900739 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:14.900739 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:14.900739 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:14.900739 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:14.900739 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:14.900739 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:14.904194 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:14.904194 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 18 06:07:14.904194 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 18 06:07:14.904194 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:14.904194 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:14.904194 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 18 06:07:14.904194 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 18 06:07:14.904194 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0a70a48a87dd46cbb6a3f593325e08f5. Jul 18 06:07:14.904194 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:14.904194 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 453/933] 10.222.0.40 () {60 vars in 1140 bytes} [Mon Jul 18 06:07:14 2022] POST /identity/v3/projects => generated 262 bytes in 297 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:07:14.904194 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c54db-41ce-4ceb-bede-4c5c4cf73d61 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:14.928777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca97914a-e386-410f-823c-064ea8670f46 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:14.998685 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f97a32-570d-468c-9387-11862a3fe61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:14.999779 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f97a32-570d-468c-9387-11862a3fe61b None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:15.000327 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f97a32-570d-468c-9387-11862a3fe61b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:15.001085 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f97a32-570d-468c-9387-11862a3fe61b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:15.003195 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f97a32-570d-468c-9387-11862a3fe61b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:15.004912 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f97a32-570d-468c-9387-11862a3fe61b None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:15.056806 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c54db-41ce-4ceb-bede-4c5c4cf73d61 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:15.104539 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 481/934] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Jul 18 06:07:14 2022] POST /identity/v3/endpoints => generated 347 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:07:15.139083 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a2ed0-1567-4b34-8dfb-331f302f12c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:15.140371 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a2ed0-1567-4b34-8dfb-331f302f12c1 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:15.140808 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a2ed0-1567-4b34-8dfb-331f302f12c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:15.141112 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a2ed0-1567-4b34-8dfb-331f302f12c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:15.154330 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15a2ed0-1567-4b34-8dfb-331f302f12c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:15.157145 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15a2ed0-1567-4b34-8dfb-331f302f12c1 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:15.184024 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-f15a2ed0-1567-4b34-8dfb-331f302f12c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:07:15.184024 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:15.184024 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:15.184024 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:15.184024 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:15.184024 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:15.184024 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:15.184024 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:15.184024 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:15.184024 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:15.184024 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:15.184024 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:15.184024 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 06:07:15.184024 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 06:07:15.184024 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 06:07:15.185487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 06:07:15.185487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:15.185487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:15.185487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:15.185487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:15.185487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:15.185487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:15.185487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:15.185487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:15.185487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:15.185487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:15.185487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:15.185487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:15.185487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:15.185487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:15.185487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:15.185487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:15.187031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 06:07:15.187031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 06:07:15.187031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 06:07:15.187031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 06:07:15.187031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 06:07:15.187031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 06:07:15.187031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:07:15.187031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:15.187031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 454/935] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:07:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:15.195401 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f97a32-570d-468c-9387-11862a3fe61b None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:15.382517 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:15.401350 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f97a32-570d-468c-9387-11862a3fe61b None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:15.475512 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e8d7b-5181-47af-aba9-91d00312eccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:15.477845 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e8d7b-5181-47af-aba9-91d00312eccc None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:15.478333 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e8d7b-5181-47af-aba9-91d00312eccc None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:15.478937 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e8d7b-5181-47af-aba9-91d00312eccc None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:15.480504 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e8d7b-5181-47af-aba9-91d00312eccc None admin] RBAC: Adding query filter params (name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:15.481010 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e8d7b-5181-47af-aba9-91d00312eccc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:15.484561 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e8d7b-5181-47af-aba9-91d00312eccc None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:15.519687 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 482/936] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Jul 18 06:07:15 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:15.952697 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a411ea-4a5a-4fba-b171-108c339259b6 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:15.953433 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a411ea-4a5a-4fba-b171-108c339259b6 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:15.953433 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a411ea-4a5a-4fba-b171-108c339259b6 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:15.954810 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 455/937] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:07:15.961508 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90ed5f-92d4-43da-a03f-077d678cee32 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:15.961887 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90ed5f-92d4-43da-a03f-077d678cee32 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:15.962129 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90ed5f-92d4-43da-a03f-077d678cee32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:15.981503 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-cc90ed5f-92d4-43da-a03f-077d678cee32 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:15.982723 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-cc90ed5f-92d4-43da-a03f-077d678cee32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:16.111135 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 483/938] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 06:07:16.134923 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a411ea-4a5a-4fba-b171-108c339259b6 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:16.154698 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:16.267256 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a411ea-4a5a-4fba-b171-108c339259b6 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:16.325091 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c0be0e-7164-4dfd-a0df-5ac3cfdecc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:16.326407 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0be0e-7164-4dfd-a0df-5ac3cfdecc5e None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:16.326911 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0be0e-7164-4dfd-a0df-5ac3cfdecc5e None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:16.327379 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0be0e-7164-4dfd-a0df-5ac3cfdecc5e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:16.328556 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c0be0e-7164-4dfd-a0df-5ac3cfdecc5e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:16.331271 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c0be0e-7164-4dfd-a0df-5ac3cfdecc5e None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:16.352440 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-26c0be0e-7164-4dfd-a0df-5ac3cfdecc5e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 06:07:16.352440 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:16.352440 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 06:07:16.352440 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 06:07:16.352440 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 06:07:16.352440 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 06:07:16.352440 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 06:07:16.352440 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:07:16.352440 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 06:07:16.352440 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:16.352440 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 06:07:16.352440 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:16.352440 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:16.352440 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:16.352440 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:16.352440 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:16.352440 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:16.355244 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:16.355244 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:16.355244 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:16.355244 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:16.355244 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:16.355244 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:16.355244 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 06:07:16.355244 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 06:07:16.355244 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 06:07:16.355244 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 06:07:16.355244 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:16.355244 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:16.355244 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:16.355244 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:16.355244 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:16.355244 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:16.358579 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:16.358579 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:16.358579 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:16.358579 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:16.358579 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:16.358579 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:16.358579 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:16.358579 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:16.358579 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:16.358579 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:16.358579 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 06:07:16.358579 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:07:16.358579 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 06:07:16.358579 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:16.358579 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 456/939] 10.222.0.40 () {56 vars in 1126 bytes} [Mon Jul 18 06:07:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:16.366246 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90ed5f-92d4-43da-a03f-077d678cee32 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:16.524928 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:16.561346 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90ed5f-92d4-43da-a03f-077d678cee32 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:16.623420 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14eeaf2-aa5e-4d13-81f6-e99f261cef7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:16.624547 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14eeaf2-aa5e-4d13-81f6-e99f261cef7d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:16.625032 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14eeaf2-aa5e-4d13-81f6-e99f261cef7d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:16.625498 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14eeaf2-aa5e-4d13-81f6-e99f261cef7d None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:16.626784 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14eeaf2-aa5e-4d13-81f6-e99f261cef7d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:16.627335 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14eeaf2-aa5e-4d13-81f6-e99f261cef7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:16.629918 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14eeaf2-aa5e-4d13-81f6-e99f261cef7d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:16.636432 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f94312-aad7-4f1d-92db-6a00fca0fb7c None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:16.636935 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f94312-aad7-4f1d-92db-6a00fca0fb7c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:16.637426 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f94312-aad7-4f1d-92db-6a00fca0fb7c None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:16.638090 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 484/940] 10.222.0.40 () {56 vars in 1122 bytes} [Mon Jul 18 06:07:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:16.640617 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 457/941] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:07:16.645830 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14eeaf2-aa5e-4d13-81f6-e99f261cef7d None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:16.653615 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf96476-3eee-472a-a229-dfc17668d817 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:16.653615 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf96476-3eee-472a-a229-dfc17668d817 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:16.653615 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf96476-3eee-472a-a229-dfc17668d817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:16.693197 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-0bf96476-3eee-472a-a229-dfc17668d817 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:16.698397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-0bf96476-3eee-472a-a229-dfc17668d817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:16.802878 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14eeaf2-aa5e-4d13-81f6-e99f261cef7d None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:16.848099 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29adeca1-4222-4b82-bd53-fc45eb6e315a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:16.849226 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29adeca1-4222-4b82-bd53-fc45eb6e315a None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:16.849694 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29adeca1-4222-4b82-bd53-fc45eb6e315a None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:16.850122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29adeca1-4222-4b82-bd53-fc45eb6e315a None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:16.850993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29adeca1-4222-4b82-bd53-fc45eb6e315a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:16.852097 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29adeca1-4222-4b82-bd53-fc45eb6e315a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:16.854014 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29adeca1-4222-4b82-bd53-fc45eb6e315a None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:16.860570 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 458/942] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:07:16.868774 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 485/943] 10.222.0.40 () {56 vars in 1122 bytes} [Mon Jul 18 06:07:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:16.874856 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf96476-3eee-472a-a229-dfc17668d817 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:16.887217 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:16.902300 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29adeca1-4222-4b82-bd53-fc45eb6e315a None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:16.927047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:17.018251 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf96476-3eee-472a-a229-dfc17668d817 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:17.041488 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29adeca1-4222-4b82-bd53-fc45eb6e315a None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:17.073162 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a108b-539f-41a5-9d00-0212e04349ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:17.073763 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a108b-539f-41a5-9d00-0212e04349ea None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:17.074382 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a108b-539f-41a5-9d00-0212e04349ea None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:17.074668 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a108b-539f-41a5-9d00-0212e04349ea None admin] PATH_INFO: `/v3/endpoints` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:17.075461 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131a108b-539f-41a5-9d00-0212e04349ea None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:17.076949 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131a108b-539f-41a5-9d00-0212e04349ea None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:17.092440 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e1070-2c15-4f82-bf80-964b4bdbcc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:17.093415 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e1070-2c15-4f82-bf80-964b4bdbcc8a None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:17.093800 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e1070-2c15-4f82-bf80-964b4bdbcc8a None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:17.094153 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e1070-2c15-4f82-bf80-964b4bdbcc8a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:17.108263 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e1070-2c15-4f82-bf80-964b4bdbcc8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:07:17.117137 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-be6e1070-2c15-4f82-bf80-964b4bdbcc8a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 06:07:17.117137 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:17.117137 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:17.117137 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:17.117137 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:17.117137 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:17.117137 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:17.117137 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:17.117137 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:17.117137 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:17.117137 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:17.117137 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:17.117137 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:07:17.117137 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:07:17.117137 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:07:17.117137 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:07:17.118180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:07:17.118180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:07:17.118180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:07:17.118180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:07:17.118180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:17.118180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:17.118180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:17.118180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:17.118180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:17.118180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:17.118180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:17.118180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:17.118180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:17.118180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:17.118180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:17.118180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:17.119056 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:17.119056 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:17.119056 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:17.119056 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:17.119056 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:07:17.119056 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:07:17.119056 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:07:17.119056 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:07:17.119056 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:07:17.119056 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:07:17.119056 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 06:07:17.119056 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:17.120382 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 486/944] 10.222.0.40 () {56 vars in 1120 bytes} [Mon Jul 18 06:07:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:07:17.125067 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e1070-2c15-4f82-bf80-964b4bdbcc8a None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:17.176228 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 459/945] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Jul 18 06:07:16 2022] POST /identity/v3/endpoints => generated 349 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:07:17.212614 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:17.255646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e1070-2c15-4f82-bf80-964b4bdbcc8a None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:17.308286 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea30022e-5d1b-4de9-82c8-e507080d64a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:17.309541 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30022e-5d1b-4de9-82c8-e507080d64a6 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:17.310053 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30022e-5d1b-4de9-82c8-e507080d64a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:17.310585 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30022e-5d1b-4de9-82c8-e507080d64a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:17.311584 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea30022e-5d1b-4de9-82c8-e507080d64a6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:17.312135 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea30022e-5d1b-4de9-82c8-e507080d64a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:17.313803 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea30022e-5d1b-4de9-82c8-e507080d64a6 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:17.325357 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 487/946] 10.222.0.40 () {56 vars in 1115 bytes} [Mon Jul 18 06:07:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:07:17.332557 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a108b-539f-41a5-9d00-0212e04349ea None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:17.409339 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:17.461272 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a108b-539f-41a5-9d00-0212e04349ea None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:17.501459 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6468a90-4a2a-49a3-ab15-9169866dcbbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:17.502676 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6468a90-4a2a-49a3-ab15-9169866dcbbe None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:17.503175 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6468a90-4a2a-49a3-ab15-9169866dcbbe None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:17.503689 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6468a90-4a2a-49a3-ab15-9169866dcbbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:17.518156 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6468a90-4a2a-49a3-ab15-9169866dcbbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:17.519977 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6468a90-4a2a-49a3-ab15-9169866dcbbe None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:17.540467 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-b6468a90-4a2a-49a3-ab15-9169866dcbbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:17.540467 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:17.540467 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:17.540467 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:17.540467 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:17.540467 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:17.540467 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:17.540467 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:17.540467 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:17.540467 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:17.540467 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:17.540467 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:17.540467 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:07:17.540467 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:07:17.540467 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:07:17.541425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:07:17.541425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:17.541425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:17.541425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:17.541425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:17.541425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:17.541425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:17.541425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:17.541425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:17.541425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:17.541425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:17.541425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:17.541425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:17.541425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:17.541425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:17.541425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:17.541425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:17.542329 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:07:17.542329 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:07:17.542329 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:17.542329 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:17.544780 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 460/947] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:07:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:07:17.552617 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea30022e-5d1b-4de9-82c8-e507080d64a6 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:17.674036 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea30022e-5d1b-4de9-82c8-e507080d64a6 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:17.706395 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:17.709528 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e83b9-e394-4416-b628-25e1cc39f0c1 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:17.709829 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e83b9-e394-4416-b628-25e1cc39f0c1 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:17.710103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e83b9-e394-4416-b628-25e1cc39f0c1 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:17.712805 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 461/948] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:07:17.717246 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa278a0-4101-4704-b5ae-7184f7a7fc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:17.717959 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa278a0-4101-4704-b5ae-7184f7a7fc7f None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:17.718280 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa278a0-4101-4704-b5ae-7184f7a7fc7f None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:17.718596 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa278a0-4101-4704-b5ae-7184f7a7fc7f None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:17.719381 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa278a0-4101-4704-b5ae-7184f7a7fc7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:17.719699 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa278a0-4101-4704-b5ae-7184f7a7fc7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:17.720078 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ecb1e3-1833-49da-a453-ab7ac012791b None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:17.720375 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ecb1e3-1833-49da-a453-ab7ac012791b None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:17.726879 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ecb1e3-1833-49da-a453-ab7ac012791b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:17.727036 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa278a0-4101-4704-b5ae-7184f7a7fc7f None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:17.736339 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 488/949] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:07:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:17.744665 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa278a0-4101-4704-b5ae-7184f7a7fc7f None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:17.747680 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-28ecb1e3-1833-49da-a453-ab7ac012791b None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:17.749056 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-28ecb1e3-1833-49da-a453-ab7ac012791b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:17.847293 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 462/950] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:07:17.871073 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ecb1e3-1833-49da-a453-ab7ac012791b None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:17.871602 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa278a0-4101-4704-b5ae-7184f7a7fc7f None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:17.913432 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e84965-05b3-460f-ae26-a05dae2f53e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:17.915292 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e84965-05b3-460f-ae26-a05dae2f53e1 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:17.915801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e84965-05b3-460f-ae26-a05dae2f53e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:17.916282 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e84965-05b3-460f-ae26-a05dae2f53e1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:17.925220 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e84965-05b3-460f-ae26-a05dae2f53e1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:17.927180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e84965-05b3-460f-ae26-a05dae2f53e1 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:17.940643 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:17.945755 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-74e84965-05b3-460f-ae26-a05dae2f53e1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 06:07:17.945755 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:17.945755 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:17.945755 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:17.945755 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:17.945755 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:17.945755 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:17.945755 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:17.945755 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:17.945755 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:17.945755 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:17.945755 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:17.945755 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 06:07:17.945755 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 06:07:17.945755 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 06:07:17.945755 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 06:07:17.946787 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:17.946787 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:17.946787 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 06:07:17.946787 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:07:17.946787 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 06:07:17.946787 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:07:17.946787 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:17.946787 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:17.946787 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:17.946787 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:17.946787 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:17.946787 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:17.946787 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:17.946787 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:17.946787 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:17.946787 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:17.946787 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:17.946787 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:17.947772 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:17.947772 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:17.947772 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 06:07:17.947772 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 06:07:17.947772 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 06:07:17.947772 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 06:07:17.947772 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 06:07:17.947772 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 06:07:17.947772 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 06:07:17.947772 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:17.950661 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 489/951] 10.222.0.40 () {56 vars in 1117 bytes} [Mon Jul 18 06:07:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:17.956610 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e84965-05b3-460f-ae26-a05dae2f53e1 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:18.020284 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ecb1e3-1833-49da-a453-ab7ac012791b None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:18.046897 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:18.075412 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be64df-e1d6-4b13-b022-8fd15649b5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:18.076672 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be64df-e1d6-4b13-b022-8fd15649b5e8 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:18.077171 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be64df-e1d6-4b13-b022-8fd15649b5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:18.077904 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be64df-e1d6-4b13-b022-8fd15649b5e8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:18.078803 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54be64df-e1d6-4b13-b022-8fd15649b5e8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:18.081295 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54be64df-e1d6-4b13-b022-8fd15649b5e8 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:18.102988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-54be64df-e1d6-4b13-b022-8fd15649b5e8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 06:07:18.102988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:18.102988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 06:07:18.102988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 06:07:18.102988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 06:07:18.102988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 06:07:18.102988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 06:07:18.102988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:07:18.102988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 06:07:18.102988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:18.102988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 06:07:18.102988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:18.102988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:18.102988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:18.102988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:18.102988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:18.102988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:18.104106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:18.104106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:18.104106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:18.104106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:18.104106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:18.104106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:18.104106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 06:07:18.104106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 06:07:18.104106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 06:07:18.104106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 06:07:18.104106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:18.104106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:18.104106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:18.104106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:18.104106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:18.104106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:18.105565 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:18.105565 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:18.105565 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:18.105565 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:18.105565 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:18.105565 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:18.105565 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:18.105565 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:18.105565 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:18.105565 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:18.105565 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 06:07:18.105565 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:07:18.105565 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 06:07:18.105565 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:18.106388 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 463/952] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:07:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:07:18.111665 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be64df-e1d6-4b13-b022-8fd15649b5e8 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:18.150737 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e84965-05b3-460f-ae26-a05dae2f53e1 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:18.202150 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb8ee2-d481-424c-becd-04a3a82aa238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:18.203877 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb8ee2-d481-424c-becd-04a3a82aa238 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:18.204367 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb8ee2-d481-424c-becd-04a3a82aa238 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:18.204893 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb8ee2-d481-424c-becd-04a3a82aa238 None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:18.206680 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acb8ee2-d481-424c-becd-04a3a82aa238 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:18.207365 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acb8ee2-d481-424c-becd-04a3a82aa238 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:18.209424 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acb8ee2-d481-424c-becd-04a3a82aa238 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:18.227422 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be64df-e1d6-4b13-b022-8fd15649b5e8 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:18.236737 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 490/953] 10.222.0.40 () {56 vars in 1149 bytes} [Mon Jul 18 06:07:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:18.248442 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:18.248670 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb8ee2-d481-424c-becd-04a3a82aa238 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:18.274844 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ef699-afd2-4051-b3bb-6de04cd300ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:18.275956 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ef699-afd2-4051-b3bb-6de04cd300ed None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:18.276354 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ef699-afd2-4051-b3bb-6de04cd300ed None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:18.277172 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ef699-afd2-4051-b3bb-6de04cd300ed None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:18.277737 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5ef699-afd2-4051-b3bb-6de04cd300ed None admin] RBAC: Adding query filter params (name=compute) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:18.278298 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5ef699-afd2-4051-b3bb-6de04cd300ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:18.279942 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5ef699-afd2-4051-b3bb-6de04cd300ed None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:18.292239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 464/954] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Jul 18 06:07:18 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:18.293404 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ef699-afd2-4051-b3bb-6de04cd300ed None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:18.361302 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb8ee2-d481-424c-becd-04a3a82aa238 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:18.364240 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:18.400179 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be552e-309b-4a58-94a9-e0c1b969d3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:18.401446 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be552e-309b-4a58-94a9-e0c1b969d3af None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:18.401446 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be552e-309b-4a58-94a9-e0c1b969d3af None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:18.401663 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be552e-309b-4a58-94a9-e0c1b969d3af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:18.411373 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ef699-afd2-4051-b3bb-6de04cd300ed None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:18.413979 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7be552e-309b-4a58-94a9-e0c1b969d3af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:18.415767 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7be552e-309b-4a58-94a9-e0c1b969d3af None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:18.435322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-e7be552e-309b-4a58-94a9-e0c1b969d3af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:18.435322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:18.435322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:18.435322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:18.435322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:18.435322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:18.435322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:18.435322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:18.435322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:18.435322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:18.435322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:18.435322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:18.435322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:07:18.435322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:07:18.435322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:07:18.436356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:07:18.436356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:18.436356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:18.436356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:18.436356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:18.436356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:18.436356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:18.436356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:18.436356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:18.436356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:18.436356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:18.436356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:18.436356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:18.436356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:18.436356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:18.436356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:18.436356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:18.437302 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:07:18.437302 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:07:18.437302 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:18.437302 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:18.437302 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 491/955] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:07:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:18.442923 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be552e-309b-4a58-94a9-e0c1b969d3af None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:18.459098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f95d0-88ca-4609-8066-f0aa7bcb68bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:18.460175 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f95d0-88ca-4609-8066-f0aa7bcb68bd None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:18.461421 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f95d0-88ca-4609-8066-f0aa7bcb68bd None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:18.461903 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f95d0-88ca-4609-8066-f0aa7bcb68bd None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:18.463470 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000f95d0-88ca-4609-8066-f0aa7bcb68bd None admin] RBAC: Adding query filter params (type=compute) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:18.464123 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000f95d0-88ca-4609-8066-f0aa7bcb68bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:18.465756 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000f95d0-88ca-4609-8066-f0aa7bcb68bd None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:18.471297 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 465/956] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Jul 18 06:07:18 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:18.599316 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401d171-b88f-476c-a1d6-bc09136ee2ea None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:18.599714 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401d171-b88f-476c-a1d6-bc09136ee2ea None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:18.599889 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401d171-b88f-476c-a1d6-bc09136ee2ea None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:18.602294 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 466/957] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:07:18.608644 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e59b00-1b79-4716-833f-65d76d8590c2 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:18.608644 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e59b00-1b79-4716-833f-65d76d8590c2 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:18.608945 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e59b00-1b79-4716-833f-65d76d8590c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:18.619241 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be552e-309b-4a58-94a9-e0c1b969d3af None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:18.635729 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-70e59b00-1b79-4716-833f-65d76d8590c2 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:18.637706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-70e59b00-1b79-4716-833f-65d76d8590c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:18.667542 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc8d61-45bb-467c-b73e-e061c9719314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:18.668244 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc8d61-45bb-467c-b73e-e061c9719314 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:18.668581 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc8d61-45bb-467c-b73e-e061c9719314 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:18.668900 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc8d61-45bb-467c-b73e-e061c9719314 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:18.669690 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddc8d61-45bb-467c-b73e-e061c9719314 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:18.669971 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddc8d61-45bb-467c-b73e-e061c9719314 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:18.671442 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddc8d61-45bb-467c-b73e-e061c9719314 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:18.693109 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:18.698739 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 492/958] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:07:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:18.709384 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc8d61-45bb-467c-b73e-e061c9719314 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:18.734460 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:18.751414 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 467/959] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 06:07:18.777563 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e59b00-1b79-4716-833f-65d76d8590c2 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:18.900600 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc8d61-45bb-467c-b73e-e061c9719314 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:18.926034 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e59b00-1b79-4716-833f-65d76d8590c2 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:18.967144 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c072f-366e-4a28-bf67-9349e03ce13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:18.968026 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c072f-366e-4a28-bf67-9349e03ce13c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:18.968503 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c072f-366e-4a28-bf67-9349e03ce13c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:18.968884 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c072f-366e-4a28-bf67-9349e03ce13c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:18.977608 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282c072f-366e-4a28-bf67-9349e03ce13c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:18.979243 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282c072f-366e-4a28-bf67-9349e03ce13c None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:18.989618 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c653fd-5616-4e26-809f-0b6c10e2a979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:18.990745 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c653fd-5616-4e26-809f-0b6c10e2a979 None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:18.991231 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c653fd-5616-4e26-809f-0b6c10e2a979 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:18.992828 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c653fd-5616-4e26-809f-0b6c10e2a979 None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:18.994257 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c653fd-5616-4e26-809f-0b6c10e2a979 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:18.996884 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c653fd-5616-4e26-809f-0b6c10e2a979 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:18.997270 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-282c072f-366e-4a28-bf67-9349e03ce13c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:07:18.997270 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:18.997270 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:18.997270 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:18.997270 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:18.997270 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:18.997270 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:18.997270 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:18.997270 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:18.997270 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:18.997270 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:18.997270 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:18.997270 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 06:07:18.997270 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 06:07:18.997270 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 06:07:18.998634 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 06:07:18.998634 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:18.998634 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:18.998634 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:18.998634 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:18.998634 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:18.998634 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:18.998634 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:18.998634 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:18.998634 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:18.998634 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:18.998634 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:18.998634 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:18.998634 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:18.998634 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:18.998634 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:18.998634 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:19.000537 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 06:07:19.000537 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 06:07:19.000537 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 06:07:19.000537 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 06:07:19.000537 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 06:07:19.000537 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 06:07:19.000537 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:07:19.000537 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:19.000537 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 493/960] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:07:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:07:19.009754 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c072f-366e-4a28-bf67-9349e03ce13c None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:19.051105 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.sql.core [None req-d4c653fd-5616-4e26-809f-0b6c10e2a979 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 18 06:07:19.051105 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 18 06:07:19.051105 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [parameters: {'user_id': '0013aa045e3f4972849c6c393712bb8f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 18 06:07:19.051105 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80471) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 18 06:07:19.060147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-d4c653fd-5616-4e26-809f-0b6c10e2a979 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 18 06:07:19.060147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:19.060147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 06:07:19.060147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 06:07:19.060147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 06:07:19.060147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 06:07:19.060147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 06:07:19.060147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 06:07:19.060147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 06:07:19.060147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application conn.query(q) Jul 18 06:07:19.060147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 06:07:19.060147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 06:07:19.060147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 06:07:19.060147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application result.read() Jul 18 06:07:19.060147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 06:07:19.061307 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 06:07:19.061307 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 06:07:19.061307 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 06:07:19.061307 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 06:07:19.061307 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 06:07:19.061307 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 06:07:19.061307 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 06:07:19.061307 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 18 06:07:19.061307 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:19.061307 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 18 06:07:19.061307 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:19.061307 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:19.061307 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 18 06:07:19.061307 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 18 06:07:19.061307 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 18 06:07:19.061307 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 18 06:07:19.061307 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 06:07:19.061307 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application next(self.gen) Jul 18 06:07:19.062712 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 18 06:07:19.062712 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application yield resource Jul 18 06:07:19.062712 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 06:07:19.062712 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application next(self.gen) Jul 18 06:07:19.062712 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 18 06:07:19.062712 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self.session.rollback() Jul 18 06:07:19.062712 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 18 06:07:19.062712 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self.force_reraise() Jul 18 06:07:19.062712 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 18 06:07:19.062712 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise self.value Jul 18 06:07:19.062712 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 18 06:07:19.062712 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 18 06:07:19.062712 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 18 06:07:19.062712 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application session.commit() Jul 18 06:07:19.062712 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 18 06:07:19.062712 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 18 06:07:19.062712 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 18 06:07:19.063901 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._prepare_impl() Jul 18 06:07:19.063901 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 18 06:07:19.063901 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self.session.flush() Jul 18 06:07:19.063901 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 18 06:07:19.063901 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._flush(objects) Jul 18 06:07:19.063901 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 18 06:07:19.063901 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 18 06:07:19.063901 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 18 06:07:19.063901 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application compat.raise_( Jul 18 06:07:19.063901 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 06:07:19.063901 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception Jul 18 06:07:19.063901 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 18 06:07:19.063901 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application flush_context.execute() Jul 18 06:07:19.063901 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 18 06:07:19.063901 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rec.execute(self) Jul 18 06:07:19.063901 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 18 06:07:19.063901 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 18 06:07:19.065101 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 18 06:07:19.065101 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 18 06:07:19.065101 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 18 06:07:19.065101 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 18 06:07:19.065101 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 18 06:07:19.065101 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 18 06:07:19.065101 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 18 06:07:19.065101 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 18 06:07:19.065101 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 18 06:07:19.065101 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 18 06:07:19.065101 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 18 06:07:19.065101 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 18 06:07:19.065101 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 18 06:07:19.065101 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 18 06:07:19.065101 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 06:07:19.065101 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception Jul 18 06:07:19.066790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 06:07:19.066790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 06:07:19.066790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 06:07:19.066790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 06:07:19.066790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 06:07:19.066790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 06:07:19.066790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 06:07:19.066790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application conn.query(q) Jul 18 06:07:19.066790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 06:07:19.066790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 06:07:19.066790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 06:07:19.066790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application result.read() Jul 18 06:07:19.066790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 06:07:19.066790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 06:07:19.066790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 06:07:19.066790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 06:07:19.077236 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 06:07:19.077236 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 06:07:19.077236 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 06:07:19.077236 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 06:07:19.077236 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 18 06:07:19.077236 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 18 06:07:19.077236 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [parameters: {'user_id': '0013aa045e3f4972849c6c393712bb8f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 18 06:07:19.077236 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 18 06:07:19.077236 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:19.077236 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 06:07:19.077236 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:19.077236 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:19.077236 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:19.077236 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:19.077236 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:19.079120 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:19.079120 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:19.079120 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:19.079120 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:19.079120 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:19.079120 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:19.079120 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:19.079120 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 18 06:07:19.079120 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 18 06:07:19.079120 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:19.079120 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:19.079120 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 06:07:19.079120 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:07:19.079120 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 06:07:19.079120 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:07:19.079120 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 18 06:07:19.079120 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 18 06:07:19.085252 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 18 06:07:19.085252 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 18 06:07:19.085252 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 18 06:07:19.085252 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 18 06:07:19.085252 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 18 06:07:19.085252 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:19.085252 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 468/961] 10.222.0.40 () {60 vars in 1131 bytes} [Mon Jul 18 06:07:18 2022] POST /identity/v3/users => generated 160 bytes in 290 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:19.085252 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c653fd-5616-4e26-809f-0b6c10e2a979 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:19.111169 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:19.168498 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:19.201687 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c653fd-5616-4e26-809f-0b6c10e2a979 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:19.206272 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c072f-366e-4a28-bf67-9349e03ce13c None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:19.244972 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76049b8-192c-41d6-9818-4a8c961fb082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:19.246868 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76049b8-192c-41d6-9818-4a8c961fb082 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:19.247520 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76049b8-192c-41d6-9818-4a8c961fb082 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:19.248102 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76049b8-192c-41d6-9818-4a8c961fb082 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:19.255666 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11d4ac-30ca-4e01-a0b9-ffcdb08ed551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:19.256505 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11d4ac-30ca-4e01-a0b9-ffcdb08ed551 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:19.257000 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11d4ac-30ca-4e01-a0b9-ffcdb08ed551 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:19.257461 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11d4ac-30ca-4e01-a0b9-ffcdb08ed551 None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:19.258486 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b11d4ac-30ca-4e01-a0b9-ffcdb08ed551 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:19.258870 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b11d4ac-30ca-4e01-a0b9-ffcdb08ed551 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:19.260463 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b11d4ac-30ca-4e01-a0b9-ffcdb08ed551 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:19.260879 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76049b8-192c-41d6-9818-4a8c961fb082 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:19.263268 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76049b8-192c-41d6-9818-4a8c961fb082 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:19.280664 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 494/962] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 18 06:07:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:19.282591 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11d4ac-30ca-4e01-a0b9-ffcdb08ed551 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:19.284727 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-a76049b8-192c-41d6-9818-4a8c961fb082 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 06:07:19.284727 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:19.284727 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:19.284727 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:19.284727 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:19.284727 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:19.284727 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:19.284727 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:19.284727 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:19.284727 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:19.284727 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:19.284727 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:19.284727 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 06:07:19.284727 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 06:07:19.284727 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 06:07:19.284727 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 06:07:19.286068 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:19.286068 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:19.286068 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 06:07:19.286068 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:07:19.286068 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 06:07:19.286068 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:07:19.286068 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:19.286068 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:19.286068 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:19.286068 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:19.286068 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:19.286068 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:19.286068 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:19.286068 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:19.286068 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:19.286068 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:19.286068 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:19.286068 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:19.287032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:19.287032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:19.287032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 06:07:19.287032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 06:07:19.287032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 06:07:19.287032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 06:07:19.287032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 06:07:19.287032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 06:07:19.287032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 06:07:19.287032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:19.291888 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 469/963] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:07:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:19.297880 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76049b8-192c-41d6-9818-4a8c961fb082 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:19.409181 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:19.423322 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76049b8-192c-41d6-9818-4a8c961fb082 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:19.437413 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11d4ac-30ca-4e01-a0b9-ffcdb08ed551 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:19.472533 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23d7c1f-1615-4e34-b7fe-d18a19e930b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:19.473597 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d7c1f-1615-4e34-b7fe-d18a19e930b0 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:19.474162 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d7c1f-1615-4e34-b7fe-d18a19e930b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:19.474641 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d7c1f-1615-4e34-b7fe-d18a19e930b0 None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:19.475657 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23d7c1f-1615-4e34-b7fe-d18a19e930b0 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:19.476214 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23d7c1f-1615-4e34-b7fe-d18a19e930b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:19.478354 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23d7c1f-1615-4e34-b7fe-d18a19e930b0 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:19.494622 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ddacf2-ac05-47d2-944f-0b05c31a564d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:19.495607 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ddacf2-ac05-47d2-944f-0b05c31a564d None admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:19.495835 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ddacf2-ac05-47d2-944f-0b05c31a564d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:19.496119 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ddacf2-ac05-47d2-944f-0b05c31a564d None admin] PATH_INFO: `/v3/projects/8ed50a8a3e184bde9269d7b0d9573d4a/users/b84dde57ff8e47419866377a819bea21/roles/34f769e98e1e449cb45aa4b31836393a` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:19.498593 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ddacf2-ac05-47d2-944f-0b05c31a564d None admin] RBAC: Authorizing `identity:create_grant(project_id=8ed50a8a3e184bde9269d7b0d9573d4a, user_id=b84dde57ff8e47419866377a819bea21, role_id=34f769e98e1e449cb45aa4b31836393a)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:19.500142 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ddacf2-ac05-47d2-944f-0b05c31a564d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:19.503596 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:19.526795 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 470/964] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 18 06:07:19 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:07:19.592132 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053fa56-cc67-49de-b998-0bdb1f73caf3 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:19.592501 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053fa56-cc67-49de-b998-0bdb1f73caf3 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:19.592745 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053fa56-cc67-49de-b998-0bdb1f73caf3 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:19.594389 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 471/965] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:07:19.600148 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003c845-d468-46ed-af64-a54813560ecb None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:19.600494 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003c845-d468-46ed-af64-a54813560ecb None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:19.600754 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003c845-d468-46ed-af64-a54813560ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:19.621429 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-9003c845-d468-46ed-af64-a54813560ecb None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:19.623104 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-9003c845-d468-46ed-af64-a54813560ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:19.641328 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 495/966] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 18 06:07:19 2022] PUT /identity/v3/projects/8ed50a8a3e184bde9269d7b0d9573d4a/users/b84dde57ff8e47419866377a819bea21/roles/34f769e98e1e449cb45aa4b31836393a => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:07:19.751675 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 472/967] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 06:07:19.775930 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ddacf2-ac05-47d2-944f-0b05c31a564d None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:19.900016 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ddacf2-ac05-47d2-944f-0b05c31a564d None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:19.936217 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf1672c-1c94-4113-b033-b06a76315d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:19.937074 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1672c-1c94-4113-b033-b06a76315d65 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:19.937362 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1672c-1c94-4113-b033-b06a76315d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:19.937642 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1672c-1c94-4113-b033-b06a76315d65 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:19.938350 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf1672c-1c94-4113-b033-b06a76315d65 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:19.939759 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf1672c-1c94-4113-b033-b06a76315d65 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:19.952402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-0bf1672c-1c94-4113-b033-b06a76315d65 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 06:07:19.952402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:19.952402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 06:07:19.952402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 06:07:19.952402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 06:07:19.952402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 06:07:19.952402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 06:07:19.952402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:07:19.952402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 06:07:19.952402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:19.952402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 06:07:19.952402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:19.952402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:19.952402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:19.952402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:19.952402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:19.952402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:19.953531 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:19.953531 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:19.953531 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:19.953531 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:19.953531 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:19.953531 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:19.953531 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 06:07:19.953531 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 06:07:19.953531 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 06:07:19.953531 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 06:07:19.953531 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:19.953531 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:19.953531 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:19.953531 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:19.953531 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:19.953531 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:19.954375 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:19.954375 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:19.954375 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:19.954375 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:19.954375 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:19.954375 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:19.954375 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:19.954375 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:19.954375 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:19.954375 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:19.954375 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 06:07:19.954375 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:07:19.954375 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 06:07:19.954375 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:19.954375 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 496/968] 10.222.0.40 () {56 vars in 1126 bytes} [Mon Jul 18 06:07:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:07:19.959742 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9003c845-d468-46ed-af64-a54813560ecb None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:20.044954 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:20.067732 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9003c845-d468-46ed-af64-a54813560ecb None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:20.106191 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79c1c49-a421-4410-af8b-f74f000f5fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:20.106869 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c1c49-a421-4410-af8b-f74f000f5fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:20.107159 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c1c49-a421-4410-af8b-f74f000f5fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:20.107441 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c1c49-a421-4410-af8b-f74f000f5fe5 None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:20.108158 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79c1c49-a421-4410-af8b-f74f000f5fe5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:20.108499 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79c1c49-a421-4410-af8b-f74f000f5fe5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:20.109870 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79c1c49-a421-4410-af8b-f74f000f5fe5 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:20.119851 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 473/969] 10.222.0.40 () {56 vars in 1122 bytes} [Mon Jul 18 06:07:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:20.121787 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf1672c-1c94-4113-b033-b06a76315d65 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:20.187324 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:20.231301 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf1672c-1c94-4113-b033-b06a76315d65 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:20.266412 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac7a31-32ef-48c9-a1ca-5e705cb52895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:20.267095 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac7a31-32ef-48c9-a1ca-5e705cb52895 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:20.267385 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac7a31-32ef-48c9-a1ca-5e705cb52895 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:20.267662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac7a31-32ef-48c9-a1ca-5e705cb52895 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:20.268379 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac7a31-32ef-48c9-a1ca-5e705cb52895 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:20.268719 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac7a31-32ef-48c9-a1ca-5e705cb52895 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:20.270111 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac7a31-32ef-48c9-a1ca-5e705cb52895 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:20.282208 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 497/970] 10.222.0.40 () {56 vars in 1122 bytes} [Mon Jul 18 06:07:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:20.288237 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79c1c49-a421-4410-af8b-f74f000f5fe5 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:20.392152 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79c1c49-a421-4410-af8b-f74f000f5fe5 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:20.428781 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bd327e-f8c4-4384-ba6c-4798e7951813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:20.429481 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd327e-f8c4-4384-ba6c-4798e7951813 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:20.429803 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd327e-f8c4-4384-ba6c-4798e7951813 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:20.430083 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd327e-f8c4-4384-ba6c-4798e7951813 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:20.430895 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bd327e-f8c4-4384-ba6c-4798e7951813 None admin] RBAC: Adding query filter params (interface=admin, service_id=9e05e550ea8f4422a52f1c6dfad15750, region_id=RegionOne) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:20.431203 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bd327e-f8c4-4384-ba6c-4798e7951813 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:20.432593 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bd327e-f8c4-4384-ba6c-4798e7951813 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:20.442427 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 474/971] 10.222.0.40 () {56 vars in 1249 bytes} [Mon Jul 18 06:07:20 2022] GET /identity/v3/endpoints?service_id=9e05e550ea8f4422a52f1c6dfad15750&interface=admin®ion_id=RegionOne => generated 193 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:20.443773 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac7a31-32ef-48c9-a1ca-5e705cb52895 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:20.549943 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac7a31-32ef-48c9-a1ca-5e705cb52895 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:20.573668 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:20.585048 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84531e79-8646-48d8-a922-ae146e2a25c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:20.586403 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84531e79-8646-48d8-a922-ae146e2a25c9 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:20.597316 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84531e79-8646-48d8-a922-ae146e2a25c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:20.597316 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84531e79-8646-48d8-a922-ae146e2a25c9 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:20.597316 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84531e79-8646-48d8-a922-ae146e2a25c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:20.597316 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84531e79-8646-48d8-a922-ae146e2a25c9 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:20.597316 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 498/972] 10.222.0.40 () {56 vars in 1087 bytes} [Mon Jul 18 06:07:20 2022] GET /identity/v3/services => generated 1497 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 06:07:21.467663 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3113a-59ad-40be-87c0-a7a0b79b9845 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:21.468294 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3113a-59ad-40be-87c0-a7a0b79b9845 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:21.468294 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3113a-59ad-40be-87c0-a7a0b79b9845 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:21.469749 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 475/973] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:07:21.474924 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11995bfe-dc19-4cda-9785-a2f3f472cbee None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:21.475351 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11995bfe-dc19-4cda-9785-a2f3f472cbee None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:21.475532 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11995bfe-dc19-4cda-9785-a2f3f472cbee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:21.494214 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-11995bfe-dc19-4cda-9785-a2f3f472cbee None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:21.495464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-11995bfe-dc19-4cda-9785-a2f3f472cbee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:21.566565 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 499/974] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:07:21.588960 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3113a-59ad-40be-87c0-a7a0b79b9845 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:21.598297 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:21.704600 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3113a-59ad-40be-87c0-a7a0b79b9845 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:21.744165 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d843eb0-23c3-4e5f-9082-4b6936419598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:21.744774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d843eb0-23c3-4e5f-9082-4b6936419598 None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:21.745055 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d843eb0-23c3-4e5f-9082-4b6936419598 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:21.745330 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d843eb0-23c3-4e5f-9082-4b6936419598 None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:21.746109 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d843eb0-23c3-4e5f-9082-4b6936419598 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:21.747483 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d843eb0-23c3-4e5f-9082-4b6936419598 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:21.875361 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 476/975] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Jul 18 06:07:21 2022] POST /identity/v3/services => generated 242 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:07:22.385042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b2406-06f0-4051-8edd-c630c1102c74 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:22.385498 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b2406-06f0-4051-8edd-c630c1102c74 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:22.385944 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b2406-06f0-4051-8edd-c630c1102c74 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:22.388305 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 500/976] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:07:22.396760 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41b761-7432-42f7-a5d3-2e67a50249d9 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:22.397206 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41b761-7432-42f7-a5d3-2e67a50249d9 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:22.397637 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41b761-7432-42f7-a5d3-2e67a50249d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:22.426214 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-db41b761-7432-42f7-a5d3-2e67a50249d9 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:22.428150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-db41b761-7432-42f7-a5d3-2e67a50249d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:22.510644 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:22.530347 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 477/977] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:22 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:07:22.555329 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584b2406-06f0-4051-8edd-c630c1102c74 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:22.668988 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584b2406-06f0-4051-8edd-c630c1102c74 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:22.691344 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:22.708611 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4023c9d-f073-49fe-9b03-3bcc5b94b6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:22.709430 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4023c9d-f073-49fe-9b03-3bcc5b94b6c8 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:22.709716 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4023c9d-f073-49fe-9b03-3bcc5b94b6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:22.709997 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4023c9d-f073-49fe-9b03-3bcc5b94b6c8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:22.720304 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4023c9d-f073-49fe-9b03-3bcc5b94b6c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:07:22.729594 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-b4023c9d-f073-49fe-9b03-3bcc5b94b6c8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 06:07:22.729594 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:22.729594 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:22.729594 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:22.729594 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:22.729594 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:22.729594 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:22.729594 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:22.729594 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:22.729594 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:22.729594 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:22.729594 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:22.729594 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:07:22.729594 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:07:22.729594 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:07:22.729594 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:07:22.731064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:07:22.731064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:07:22.731064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:07:22.731064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:07:22.731064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:22.731064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:22.731064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:22.731064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:22.731064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:22.731064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:22.731064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:22.731064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:22.731064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:22.731064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:22.731064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:22.731064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:22.732295 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:22.732295 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:22.732295 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:22.732295 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:22.732295 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:07:22.732295 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:07:22.732295 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:07:22.732295 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:07:22.732295 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:07:22.732295 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:07:22.732295 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 06:07:22.732295 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:22.736157 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 501/978] 10.222.0.40 () {56 vars in 1120 bytes} [Mon Jul 18 06:07:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:22.741064 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db41b761-7432-42f7-a5d3-2e67a50249d9 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:22.856379 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db41b761-7432-42f7-a5d3-2e67a50249d9 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:22.893024 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb4329-eb27-45e7-ac64-0827c816d9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:22.893707 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb4329-eb27-45e7-ac64-0827c816d9ab None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:22.893994 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb4329-eb27-45e7-ac64-0827c816d9ab None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:22.894272 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb4329-eb27-45e7-ac64-0827c816d9ab None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:22.895034 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb4329-eb27-45e7-ac64-0827c816d9ab None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:22.895340 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb4329-eb27-45e7-ac64-0827c816d9ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:22.896730 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb4329-eb27-45e7-ac64-0827c816d9ab None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:22.911266 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 478/979] 10.222.0.40 () {56 vars in 1115 bytes} [Mon Jul 18 06:07:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:22.912708 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4023c9d-f073-49fe-9b03-3bcc5b94b6c8 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:23.026891 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4023c9d-f073-49fe-9b03-3bcc5b94b6c8 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:23.064915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a9302b-8b1a-4a67-b973-f91e2052ad1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:23.065919 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9302b-8b1a-4a67-b973-f91e2052ad1c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:23.066130 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9302b-8b1a-4a67-b973-f91e2052ad1c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:23.066379 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9302b-8b1a-4a67-b973-f91e2052ad1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:23.078836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a9302b-8b1a-4a67-b973-f91e2052ad1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:23.080317 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a9302b-8b1a-4a67-b973-f91e2052ad1c None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:23.098998 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-f7a9302b-8b1a-4a67-b973-f91e2052ad1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:23.098998 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:23.098998 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:23.098998 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:23.098998 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:23.098998 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:23.098998 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:23.098998 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:23.098998 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:23.098998 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:23.098998 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:23.098998 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:23.098998 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:07:23.098998 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:07:23.098998 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:07:23.100055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:07:23.100055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:23.100055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:23.100055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:23.100055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:23.100055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:23.100055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:23.100055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:23.100055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:23.100055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:23.100055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:23.100055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:23.100055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:23.100055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:23.100055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:23.100055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:23.100055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:23.102155 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:07:23.102155 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:07:23.102155 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:23.102155 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:23.102650 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 502/980] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:07:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:23.106564 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb4329-eb27-45e7-ac64-0827c816d9ab None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:23.167369 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:23.214984 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb4329-eb27-45e7-ac64-0827c816d9ab None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:23.260168 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b4c85-0ad7-48fb-a387-40b5182abec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:23.260893 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b4c85-0ad7-48fb-a387-40b5182abec4 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:23.261174 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b4c85-0ad7-48fb-a387-40b5182abec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:23.261453 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b4c85-0ad7-48fb-a387-40b5182abec4 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:23.262212 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81b4c85-0ad7-48fb-a387-40b5182abec4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:23.262512 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81b4c85-0ad7-48fb-a387-40b5182abec4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:23.263929 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81b4c85-0ad7-48fb-a387-40b5182abec4 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:23.283304 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 479/981] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:07:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:23.284750 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a9302b-8b1a-4a67-b973-f91e2052ad1c None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:23.318018 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:23.365339 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f0c4f-0bab-45e9-900f-89f51f3b227e None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:23.365754 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f0c4f-0bab-45e9-900f-89f51f3b227e None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:23.366073 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f0c4f-0bab-45e9-900f-89f51f3b227e None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:23.367780 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 480/982] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:07:23.373288 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f798c-0fe7-4825-b546-91e23b686ac1 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:23.373590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f798c-0fe7-4825-b546-91e23b686ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:23.373875 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f798c-0fe7-4825-b546-91e23b686ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:23.392298 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-8d1f798c-0fe7-4825-b546-91e23b686ac1 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:23.393548 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8d1f798c-0fe7-4825-b546-91e23b686ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:23.396950 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a9302b-8b1a-4a67-b973-f91e2052ad1c None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:23.435217 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879f1d6c-29f5-4602-98bb-96f6d1c83db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:23.436391 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f1d6c-29f5-4602-98bb-96f6d1c83db2 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:23.436917 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f1d6c-29f5-4602-98bb-96f6d1c83db2 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:23.437378 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f1d6c-29f5-4602-98bb-96f6d1c83db2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:23.446779 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879f1d6c-29f5-4602-98bb-96f6d1c83db2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:23.448557 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879f1d6c-29f5-4602-98bb-96f6d1c83db2 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:23.465015 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-879f1d6c-29f5-4602-98bb-96f6d1c83db2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 06:07:23.465015 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:23.465015 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:23.465015 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:23.465015 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:23.465015 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:23.465015 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:23.465015 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:23.465015 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:23.465015 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:23.465015 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:23.465015 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:23.465015 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 06:07:23.465015 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 06:07:23.465015 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 06:07:23.465015 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 06:07:23.466022 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:23.466022 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:23.466022 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 06:07:23.466022 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:07:23.466022 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 06:07:23.466022 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:07:23.466022 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:23.466022 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:23.466022 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:23.466022 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:23.466022 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:23.466022 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:23.466022 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:23.466022 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:23.466022 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:23.466022 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:23.466022 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:23.466022 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:23.467029 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:23.467029 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:23.467029 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 06:07:23.467029 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 06:07:23.467029 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 06:07:23.467029 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 06:07:23.467029 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 06:07:23.467029 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 06:07:23.467029 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 06:07:23.467029 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:23.468466 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 481/983] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:23 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 06:07:23.478810 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 503/984] 10.222.0.40 () {56 vars in 1117 bytes} [Mon Jul 18 06:07:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:23.480014 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f798c-0fe7-4825-b546-91e23b686ac1 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:23.491164 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879f1d6c-29f5-4602-98bb-96f6d1c83db2 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:23.532265 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:23.595459 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f798c-0fe7-4825-b546-91e23b686ac1 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:23.600202 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:23.611481 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879f1d6c-29f5-4602-98bb-96f6d1c83db2 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:23.632932 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4778b0d9-5612-4965-9713-64a0bdcd06ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:23.633607 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778b0d9-5612-4965-9713-64a0bdcd06ee None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:23.633897 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778b0d9-5612-4965-9713-64a0bdcd06ee None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:23.634177 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778b0d9-5612-4965-9713-64a0bdcd06ee None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:23.635035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4778b0d9-5612-4965-9713-64a0bdcd06ee None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:23.635344 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4778b0d9-5612-4965-9713-64a0bdcd06ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:23.636807 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4778b0d9-5612-4965-9713-64a0bdcd06ee None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:23.656061 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 482/985] 10.222.0.40 () {56 vars in 1149 bytes} [Mon Jul 18 06:07:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:23.661755 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4778b0d9-5612-4965-9713-64a0bdcd06ee None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:23.664800 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0e081-d104-4af4-97a4-38ad80f6c9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:23.665896 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0e081-d104-4af4-97a4-38ad80f6c9bc None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:23.667118 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0e081-d104-4af4-97a4-38ad80f6c9bc None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:23.667468 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0e081-d104-4af4-97a4-38ad80f6c9bc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:23.668153 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed0e081-d104-4af4-97a4-38ad80f6c9bc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:23.669588 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed0e081-d104-4af4-97a4-38ad80f6c9bc None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:23.686676 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-aed0e081-d104-4af4-97a4-38ad80f6c9bc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 06:07:23.686676 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:23.686676 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 06:07:23.686676 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 06:07:23.686676 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 06:07:23.686676 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 06:07:23.686676 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 06:07:23.686676 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:07:23.686676 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 06:07:23.686676 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:23.686676 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 06:07:23.686676 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:23.686676 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:23.686676 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:23.686676 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:23.686676 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:23.686676 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:23.687717 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:23.687717 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:23.687717 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:23.687717 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:23.687717 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:23.687717 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:23.687717 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 06:07:23.687717 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 06:07:23.687717 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 06:07:23.687717 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 06:07:23.687717 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:23.687717 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:23.687717 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:23.687717 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:23.687717 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:23.687717 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:23.688618 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:23.688618 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:23.688618 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:23.688618 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:23.688618 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:23.688618 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:23.688618 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:23.688618 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:23.688618 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:23.688618 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:23.688618 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 06:07:23.688618 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:07:23.688618 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 06:07:23.688618 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:23.692861 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 504/986] 10.222.0.40 () {56 vars in 1126 bytes} [Mon Jul 18 06:07:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:07:23.694347 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0e081-d104-4af4-97a4-38ad80f6c9bc None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:23.774971 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4778b0d9-5612-4965-9713-64a0bdcd06ee None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:23.778713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:23.808220 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0e081-d104-4af4-97a4-38ad80f6c9bc None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:23.815479 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75be7feb-304a-45b0-8c29-ca02d026e721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:23.816147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be7feb-304a-45b0-8c29-ca02d026e721 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:23.816455 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be7feb-304a-45b0-8c29-ca02d026e721 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:23.816731 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be7feb-304a-45b0-8c29-ca02d026e721 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:23.849664 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3453d7d-6822-4e2c-8a0b-aec46c87788d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:23.849664 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3453d7d-6822-4e2c-8a0b-aec46c87788d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:23.849664 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3453d7d-6822-4e2c-8a0b-aec46c87788d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:23.849664 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3453d7d-6822-4e2c-8a0b-aec46c87788d None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:23.850487 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3453d7d-6822-4e2c-8a0b-aec46c87788d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:23.850830 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3453d7d-6822-4e2c-8a0b-aec46c87788d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:23.853000 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3453d7d-6822-4e2c-8a0b-aec46c87788d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:23.858428 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 505/987] 10.222.0.40 () {56 vars in 1122 bytes} [Mon Jul 18 06:07:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:07:23.864208 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3453d7d-6822-4e2c-8a0b-aec46c87788d None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:23.886712 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:23.898233 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75be7feb-304a-45b0-8c29-ca02d026e721 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:23.899656 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75be7feb-304a-45b0-8c29-ca02d026e721 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:23.971363 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3453d7d-6822-4e2c-8a0b-aec46c87788d None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:24.012763 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330fa2db-99d8-4648-9ac5-02c6edfb7199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:24.013462 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330fa2db-99d8-4648-9ac5-02c6edfb7199 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:24.013876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330fa2db-99d8-4648-9ac5-02c6edfb7199 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:24.014058 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330fa2db-99d8-4648-9ac5-02c6edfb7199 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:24.014830 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330fa2db-99d8-4648-9ac5-02c6edfb7199 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:24.015142 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330fa2db-99d8-4648-9ac5-02c6edfb7199 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:24.016591 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330fa2db-99d8-4648-9ac5-02c6edfb7199 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:24.018451 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 483/988] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:07:23 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 06:07:24.019848 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e04fda-3a79-4e5b-88ec-8a0cc08628a9 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:24.020146 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e04fda-3a79-4e5b-88ec-8a0cc08628a9 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:24.020445 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e04fda-3a79-4e5b-88ec-8a0cc08628a9 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:24.021799 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 484/989] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:07:24.024722 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e04fda-3a79-4e5b-88ec-8a0cc08628a9 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:24.029033 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:24.037831 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 506/990] 10.222.0.40 () {56 vars in 1122 bytes} [Mon Jul 18 06:07:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:07:24.037831 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3758d0-4089-40fb-99a9-c19c73286acd None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:24.037831 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3758d0-4089-40fb-99a9-c19c73286acd None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:24.037831 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3758d0-4089-40fb-99a9-c19c73286acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:24.053453 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-5d3758d0-4089-40fb-99a9-c19c73286acd None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:24.055520 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-5d3758d0-4089-40fb-99a9-c19c73286acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:24.132859 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 507/991] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:24 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:07:24.134310 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3758d0-4089-40fb-99a9-c19c73286acd None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:24.149765 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e04fda-3a79-4e5b-88ec-8a0cc08628a9 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:24.150854 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:24.193664 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624eb0ad-5e6c-4b5f-bed0-2be907a7a5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:24.194500 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624eb0ad-5e6c-4b5f-bed0-2be907a7a5e9 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:24.195391 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624eb0ad-5e6c-4b5f-bed0-2be907a7a5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:24.195648 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624eb0ad-5e6c-4b5f-bed0-2be907a7a5e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:24.209082 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624eb0ad-5e6c-4b5f-bed0-2be907a7a5e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:24.211186 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624eb0ad-5e6c-4b5f-bed0-2be907a7a5e9 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:24.232493 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-624eb0ad-5e6c-4b5f-bed0-2be907a7a5e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:24.232493 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:24.232493 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:24.232493 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:24.232493 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:24.232493 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:24.232493 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:24.232493 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:24.232493 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:24.232493 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:24.232493 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:24.232493 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:24.232493 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:07:24.232493 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:07:24.232493 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:07:24.233508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:07:24.233508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:24.233508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:24.233508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:24.233508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:24.233508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:24.233508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:24.233508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:24.233508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:24.233508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:24.233508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:24.233508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:24.233508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:24.233508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:24.233508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:24.233508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:24.233508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:24.234403 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:07:24.234403 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:07:24.234403 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:24.234403 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:24.234889 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 485/992] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:07:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:24.236047 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624eb0ad-5e6c-4b5f-bed0-2be907a7a5e9 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:24.254425 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3758d0-4089-40fb-99a9-c19c73286acd None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:24.294140 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b2acd-ba81-4646-af7d-01690af1cf11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:24.294812 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b2acd-ba81-4646-af7d-01690af1cf11 None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:24.295096 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b2acd-ba81-4646-af7d-01690af1cf11 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:24.295372 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b2acd-ba81-4646-af7d-01690af1cf11 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:24.296164 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4b2acd-ba81-4646-af7d-01690af1cf11 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:24.297583 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4b2acd-ba81-4646-af7d-01690af1cf11 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:24.300605 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:24.346777 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624eb0ad-5e6c-4b5f-bed0-2be907a7a5e9 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:24.384099 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730c323-66b4-400a-a197-4590b0cc94b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:24.384837 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730c323-66b4-400a-a197-4590b0cc94b7 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:24.385119 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730c323-66b4-400a-a197-4590b0cc94b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:24.385393 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730c323-66b4-400a-a197-4590b0cc94b7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:24.386048 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730c323-66b4-400a-a197-4590b0cc94b7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:24.387468 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730c323-66b4-400a-a197-4590b0cc94b7 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:24.394747 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 508/993] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Jul 18 06:07:24 2022] POST /identity/v3/endpoints => generated 325 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:07:24.395609 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b2acd-ba81-4646-af7d-01690af1cf11 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:24.399766 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-2730c323-66b4-400a-a197-4590b0cc94b7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 06:07:24.399766 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:24.399766 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 06:07:24.399766 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 06:07:24.399766 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 06:07:24.399766 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 06:07:24.399766 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 06:07:24.399766 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:07:24.399766 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 06:07:24.399766 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:24.399766 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 06:07:24.399766 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:24.399766 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:24.399766 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:24.399766 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:24.399766 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:24.399766 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:24.412031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:24.412031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:24.412031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:24.412031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:24.412031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:24.412031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:24.412031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 06:07:24.412031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 06:07:24.412031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 06:07:24.412031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 06:07:24.412031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:24.412031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:24.412031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:24.412031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:24.412031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:24.412031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:24.412935 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:24.412935 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:24.412935 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:24.412935 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:24.412935 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:24.412935 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:24.412935 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:24.412935 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:24.412935 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:24.412935 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:24.412935 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 06:07:24.412935 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:07:24.412935 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 06:07:24.412935 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:24.412935 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 486/994] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:07:24 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:24.414423 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730c323-66b4-400a-a197-4590b0cc94b7 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:24.492644 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:24.538986 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b2acd-ba81-4646-af7d-01690af1cf11 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:24.541091 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730c323-66b4-400a-a197-4590b0cc94b7 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:24.566867 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:24.585136 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca97a3b-5e15-4b55-a4aa-ce8253e20592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:24.586336 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca97a3b-5e15-4b55-a4aa-ce8253e20592 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:24.587560 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca97a3b-5e15-4b55-a4aa-ce8253e20592 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:24.587560 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca97a3b-5e15-4b55-a4aa-ce8253e20592 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:24.587731 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a40fb7-e301-47b2-9325-e00ba20b158e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:24.587820 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca97a3b-5e15-4b55-a4aa-ce8253e20592 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:24.587904 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca97a3b-5e15-4b55-a4aa-ce8253e20592 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:24.588102 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a40fb7-e301-47b2-9325-e00ba20b158e None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:24.588824 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a40fb7-e301-47b2-9325-e00ba20b158e None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:24.588824 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a40fb7-e301-47b2-9325-e00ba20b158e None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:24.589305 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a40fb7-e301-47b2-9325-e00ba20b158e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:24.589416 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca97a3b-5e15-4b55-a4aa-ce8253e20592 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:24.590245 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a40fb7-e301-47b2-9325-e00ba20b158e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:24.592164 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a40fb7-e301-47b2-9325-e00ba20b158e None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:24.597204 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 487/995] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Jul 18 06:07:24 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:07:24.603771 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a40fb7-e301-47b2-9325-e00ba20b158e None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:24.606974 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 509/996] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:07:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:24.613198 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca97a3b-5e15-4b55-a4aa-ce8253e20592 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:24.728674 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca97a3b-5e15-4b55-a4aa-ce8253e20592 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:24.729127 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a40fb7-e301-47b2-9325-e00ba20b158e None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:24.760176 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:24.768173 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce93f9-828e-49b3-b2a6-5eda143d4d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:24.771970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3be2e3f-84a1-4c60-ac5a-1ea53fe2be9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:24.772089 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce93f9-828e-49b3-b2a6-5eda143d4d7b None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:24.772089 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce93f9-828e-49b3-b2a6-5eda143d4d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:24.772244 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be2e3f-84a1-4c60-ac5a-1ea53fe2be9c None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:24.772337 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce93f9-828e-49b3-b2a6-5eda143d4d7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:24.772446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be2e3f-84a1-4c60-ac5a-1ea53fe2be9c None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:24.772446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be2e3f-84a1-4c60-ac5a-1ea53fe2be9c None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:24.772446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3be2e3f-84a1-4c60-ac5a-1ea53fe2be9c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:24.772446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3be2e3f-84a1-4c60-ac5a-1ea53fe2be9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:24.772446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3be2e3f-84a1-4c60-ac5a-1ea53fe2be9c None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:24.778228 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ce93f9-828e-49b3-b2a6-5eda143d4d7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:24.778730 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 488/997] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Jul 18 06:07:24 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:07:24.779552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ce93f9-828e-49b3-b2a6-5eda143d4d7b None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:24.784871 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3be2e3f-84a1-4c60-ac5a-1ea53fe2be9c None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:24.795161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-d1ce93f9-828e-49b3-b2a6-5eda143d4d7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:07:24.795161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:24.795161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:24.795161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:24.795161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:24.795161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:24.795161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:24.795161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:24.795161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:24.795161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:24.795161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:24.795161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:24.795161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 06:07:24.795161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 06:07:24.795161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 06:07:24.796161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 06:07:24.796161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:24.796161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:24.796161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:24.796161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:24.796161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:24.796161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:24.796161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:24.796161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:24.796161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:24.796161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:24.796161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:24.796161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:24.796161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:24.796161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:24.796161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:24.796161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:24.797586 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 06:07:24.797586 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 06:07:24.797586 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 06:07:24.797586 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 06:07:24.797586 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 06:07:24.797586 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 06:07:24.797586 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:07:24.797586 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:24.802877 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 510/998] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:07:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:07:24.803806 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce93f9-828e-49b3-b2a6-5eda143d4d7b None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:24.840639 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:24.894986 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3be2e3f-84a1-4c60-ac5a-1ea53fe2be9c None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:24.926682 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce93f9-828e-49b3-b2a6-5eda143d4d7b None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:24.933279 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e17d58-9a48-4186-8661-aa472d3e5731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:24.933945 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e17d58-9a48-4186-8661-aa472d3e5731 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:24.934231 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e17d58-9a48-4186-8661-aa472d3e5731 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:24.934633 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e17d58-9a48-4186-8661-aa472d3e5731 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:24.935427 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e17d58-9a48-4186-8661-aa472d3e5731 None admin] RBAC: Adding query filter params (interface=public, service_id=1c9fadb7c7e442a88e42ce3cade59847, region_id=RegionOne) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:24.935740 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e17d58-9a48-4186-8661-aa472d3e5731 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:24.937188 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e17d58-9a48-4186-8661-aa472d3e5731 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:24.944452 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 489/999] 10.222.0.40 () {56 vars in 1251 bytes} [Mon Jul 18 06:07:24 2022] GET /identity/v3/endpoints?service_id=1c9fadb7c7e442a88e42ce3cade59847&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:24.948519 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e17d58-9a48-4186-8661-aa472d3e5731 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:24.971025 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc921fd-b1b3-4eb3-b91e-8b1881374ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:24.971702 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc921fd-b1b3-4eb3-b91e-8b1881374ebd None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:24.971992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc921fd-b1b3-4eb3-b91e-8b1881374ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:24.972284 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc921fd-b1b3-4eb3-b91e-8b1881374ebd None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:24.973112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc921fd-b1b3-4eb3-b91e-8b1881374ebd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:24.973544 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc921fd-b1b3-4eb3-b91e-8b1881374ebd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:24.975163 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc921fd-b1b3-4eb3-b91e-8b1881374ebd None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:24.993500 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 511/1000] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 18 06:07:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:24.994560 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc921fd-b1b3-4eb3-b91e-8b1881374ebd None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:25.050258 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:25.059956 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e17d58-9a48-4186-8661-aa472d3e5731 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:25.071509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:25.099180 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e469d2-9acf-4b27-8be7-8955b74962f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:25.100144 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e469d2-9acf-4b27-8be7-8955b74962f9 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:25.100616 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e469d2-9acf-4b27-8be7-8955b74962f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:25.101093 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e469d2-9acf-4b27-8be7-8955b74962f9 None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:25.101923 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e469d2-9acf-4b27-8be7-8955b74962f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:25.103447 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e469d2-9acf-4b27-8be7-8955b74962f9 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:25.110250 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 490/1001] 10.222.0.40 () {56 vars in 1087 bytes} [Mon Jul 18 06:07:24 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 06:07:25.123089 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc921fd-b1b3-4eb3-b91e-8b1881374ebd None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:25.166301 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9bf9e8-9017-4791-a62f-2500bba427e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:25.166997 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9bf9e8-9017-4791-a62f-2500bba427e0 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:25.167287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9bf9e8-9017-4791-a62f-2500bba427e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:25.167568 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9bf9e8-9017-4791-a62f-2500bba427e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:25.168472 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9bf9e8-9017-4791-a62f-2500bba427e0 None admin] RBAC: Adding query filter params (role.id=34f769e98e1e449cb45aa4b31836393a, scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a, user.id=b84dde57ff8e47419866377a819bea21) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:25.168797 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9bf9e8-9017-4791-a62f-2500bba427e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:25.170408 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9bf9e8-9017-4791-a62f-2500bba427e0 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:25.184477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 512/1002] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:07:24 2022] GET /identity/v3/role_assignments?user.id=b84dde57ff8e47419866377a819bea21&scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a&role.id=34f769e98e1e449cb45aa4b31836393a => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:26.501185 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53877a20-9c7c-4d5b-bf0d-ed28582588d1 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:26.501734 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53877a20-9c7c-4d5b-bf0d-ed28582588d1 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:26.501734 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53877a20-9c7c-4d5b-bf0d-ed28582588d1 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:26.503306 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 491/1003] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:07:26.510048 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2208fe-ce4d-45e0-87ab-1912fe6bf4d6 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:26.510669 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2208fe-ce4d-45e0-87ab-1912fe6bf4d6 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:26.510669 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2208fe-ce4d-45e0-87ab-1912fe6bf4d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:26.528399 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-ce2208fe-ce4d-45e0-87ab-1912fe6bf4d6 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:26.529610 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ce2208fe-ce4d-45e0-87ab-1912fe6bf4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:26.600341 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 513/1004] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:26 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:07:26.623144 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53877a20-9c7c-4d5b-bf0d-ed28582588d1 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:26.729635 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53877a20-9c7c-4d5b-bf0d-ed28582588d1 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:26.763935 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bc3f82-183a-4c7f-98fc-e3f0583c67e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:26.764676 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc3f82-183a-4c7f-98fc-e3f0583c67e0 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:26.764960 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc3f82-183a-4c7f-98fc-e3f0583c67e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:26.765230 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc3f82-183a-4c7f-98fc-e3f0583c67e0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:26.766450 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bc3f82-183a-4c7f-98fc-e3f0583c67e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:26.767919 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bc3f82-183a-4c7f-98fc-e3f0583c67e0 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:26.769907 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 492/1005] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:07:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:26.776352 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2208fe-ce4d-45e0-87ab-1912fe6bf4d6 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:26.831143 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:26.894215 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2208fe-ce4d-45e0-87ab-1912fe6bf4d6 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:26.930908 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182c58fb-cef1-443d-a263-2ec6b574c187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:26.931585 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c58fb-cef1-443d-a263-2ec6b574c187 None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:26.931877 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c58fb-cef1-443d-a263-2ec6b574c187 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:26.932162 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c58fb-cef1-443d-a263-2ec6b574c187 None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:26.932941 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182c58fb-cef1-443d-a263-2ec6b574c187 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:26.934389 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182c58fb-cef1-443d-a263-2ec6b574c187 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:27.057289 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 514/1006] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Jul 18 06:07:26 2022] POST /identity/v3/projects => generated 311 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:07:27.214740 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe709e18-71d4-4c34-b98c-d0f27d88806d None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:27.215057 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe709e18-71d4-4c34-b98c-d0f27d88806d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:27.215317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe709e18-71d4-4c34-b98c-d0f27d88806d None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:27.216751 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 493/1007] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:07:27.222208 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0048bc66-4748-42f9-94fd-c5a66f9104ed None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:27.222518 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0048bc66-4748-42f9-94fd-c5a66f9104ed None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:27.222763 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0048bc66-4748-42f9-94fd-c5a66f9104ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:27.242071 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-0048bc66-4748-42f9-94fd-c5a66f9104ed None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:27.243334 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-0048bc66-4748-42f9-94fd-c5a66f9104ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:27.289622 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76f826-4f4f-440b-8256-3d48536c55b9 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:27.289899 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76f826-4f4f-440b-8256-3d48536c55b9 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:27.290178 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76f826-4f4f-440b-8256-3d48536c55b9 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:27.291727 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 494/1008] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:07:27.296852 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ecc2cc-044d-4e51-8e43-021034d9247c None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:27.297158 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ecc2cc-044d-4e51-8e43-021034d9247c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:27.297428 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ecc2cc-044d-4e51-8e43-021034d9247c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:27.314771 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-c5ecc2cc-044d-4e51-8e43-021034d9247c None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:27.315921 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-c5ecc2cc-044d-4e51-8e43-021034d9247c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:27.328676 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 515/1009] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:27 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:07:27.351842 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0048bc66-4748-42f9-94fd-c5a66f9104ed None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:27.359587 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:27.379103 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:27.395057 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 495/1010] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:27 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 06:07:27.419158 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ecc2cc-044d-4e51-8e43-021034d9247c None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:27.459445 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0048bc66-4748-42f9-94fd-c5a66f9104ed None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:27.507179 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56f769-bb5d-4a40-8b0c-6a6d1c126458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:27.507997 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56f769-bb5d-4a40-8b0c-6a6d1c126458 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:27.508289 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56f769-bb5d-4a40-8b0c-6a6d1c126458 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:27.508602 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56f769-bb5d-4a40-8b0c-6a6d1c126458 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:27.509274 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df56f769-bb5d-4a40-8b0c-6a6d1c126458 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:27.510710 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df56f769-bb5d-4a40-8b0c-6a6d1c126458 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:27.523853 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-df56f769-bb5d-4a40-8b0c-6a6d1c126458 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 06:07:27.523853 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:27.523853 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 06:07:27.523853 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 06:07:27.523853 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 06:07:27.523853 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 06:07:27.523853 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 06:07:27.523853 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:07:27.523853 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 06:07:27.523853 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:27.523853 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 06:07:27.523853 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:27.523853 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:27.523853 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:27.523853 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:27.523853 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:27.523853 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:27.524963 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:27.524963 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:27.524963 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:27.524963 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:27.524963 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:27.524963 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:27.524963 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 06:07:27.524963 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 06:07:27.524963 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 06:07:27.524963 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 06:07:27.524963 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:27.524963 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:27.524963 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:27.524963 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:27.524963 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:27.524963 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:27.525994 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:27.525994 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:27.525994 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:27.525994 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:27.525994 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:27.525994 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:27.525994 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:27.525994 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:27.525994 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:27.525994 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:27.525994 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 06:07:27.525994 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:07:27.525994 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 06:07:27.525994 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:27.525994 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 516/1011] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:07:27 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:07:27.531247 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56f769-bb5d-4a40-8b0c-6a6d1c126458 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:27.538047 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ecc2cc-044d-4e51-8e43-021034d9247c None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:27.578075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a25f2d-e3dc-4ffb-8eca-8ceb4c77f1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:27.578792 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a25f2d-e3dc-4ffb-8eca-8ceb4c77f1ac None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:27.579105 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a25f2d-e3dc-4ffb-8eca-8ceb4c77f1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:27.579385 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a25f2d-e3dc-4ffb-8eca-8ceb4c77f1ac None admin] PATH_INFO: `/v3/services/image` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:27.580047 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a25f2d-e3dc-4ffb-8eca-8ceb4c77f1ac None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:27.581462 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a25f2d-e3dc-4ffb-8eca-8ceb4c77f1ac None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:27.594398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-95a25f2d-e3dc-4ffb-8eca-8ceb4c77f1ac None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 06:07:27.594398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:27.594398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 06:07:27.594398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 06:07:27.594398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 06:07:27.594398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 06:07:27.594398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 06:07:27.594398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:07:27.594398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 06:07:27.594398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:27.594398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 06:07:27.594398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:27.594398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:27.594398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:27.594398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:27.594398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:27.594398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:27.596988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:27.596988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:27.596988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:27.596988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:27.596988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:27.596988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:27.596988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 06:07:27.596988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 06:07:27.596988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 06:07:27.596988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 06:07:27.596988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:27.596988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:27.596988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:27.596988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:27.596988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:27.596988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:27.598617 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:27.598617 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:27.598617 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:27.598617 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:27.598617 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:27.598617 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:27.598617 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:27.598617 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:27.598617 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:27.598617 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:27.598617 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 06:07:27.598617 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:07:27.598617 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 06:07:27.598617 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:27.598617 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 496/1012] 10.222.0.40 () {56 vars in 1105 bytes} [Mon Jul 18 06:07:27 2022] GET /identity/v3/services/image => generated 86 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:07:27.601731 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a25f2d-e3dc-4ffb-8eca-8ceb4c77f1ac None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:27.641717 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:27.644876 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:27.658071 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56f769-bb5d-4a40-8b0c-6a6d1c126458 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:27.704664 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addbd820-2698-4c9e-8106-9ddd76bd1b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:27.705358 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbd820-2698-4c9e-8106-9ddd76bd1b72 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:27.705653 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbd820-2698-4c9e-8106-9ddd76bd1b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:27.705938 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbd820-2698-4c9e-8106-9ddd76bd1b72 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:27.712164 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addbd820-2698-4c9e-8106-9ddd76bd1b72 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:27.712164 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addbd820-2698-4c9e-8106-9ddd76bd1b72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:27.712164 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addbd820-2698-4c9e-8106-9ddd76bd1b72 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:27.714982 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 517/1013] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Jul 18 06:07:27 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:27.720711 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addbd820-2698-4c9e-8106-9ddd76bd1b72 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:27.729606 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a25f2d-e3dc-4ffb-8eca-8ceb4c77f1ac None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:27.769068 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dc9f7d-febe-4f1f-ac9d-3760b326872d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:27.769749 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc9f7d-febe-4f1f-ac9d-3760b326872d None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:27.770039 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc9f7d-febe-4f1f-ac9d-3760b326872d None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:27.770318 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc9f7d-febe-4f1f-ac9d-3760b326872d None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:27.771062 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dc9f7d-febe-4f1f-ac9d-3760b326872d None admin] RBAC: Adding query filter params (name=image) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:27.771376 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dc9f7d-febe-4f1f-ac9d-3760b326872d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:27.773329 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dc9f7d-febe-4f1f-ac9d-3760b326872d None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:27.778542 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 497/1014] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:07:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:27.784720 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dc9f7d-febe-4f1f-ac9d-3760b326872d None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:27.837503 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addbd820-2698-4c9e-8106-9ddd76bd1b72 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:27.881157 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96940ed8-a65d-4da3-ab4a-3a0aea4b421b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:27.882632 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96940ed8-a65d-4da3-ab4a-3a0aea4b421b None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:27.883066 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96940ed8-a65d-4da3-ab4a-3a0aea4b421b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:27.883544 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96940ed8-a65d-4da3-ab4a-3a0aea4b421b None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:27.884260 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96940ed8-a65d-4da3-ab4a-3a0aea4b421b None admin] RBAC: Adding query filter params (type=compute) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:27.884597 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96940ed8-a65d-4da3-ab4a-3a0aea4b421b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:27.886140 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96940ed8-a65d-4da3-ab4a-3a0aea4b421b None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:27.896155 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 518/1015] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Jul 18 06:07:27 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:27.897350 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96940ed8-a65d-4da3-ab4a-3a0aea4b421b None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:27.905635 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:27.908827 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dc9f7d-febe-4f1f-ac9d-3760b326872d None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:27.919313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:27.950834 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aac694-5376-435f-860c-abd4211b5f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:27.951508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aac694-5376-435f-860c-abd4211b5f41 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:27.951798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aac694-5376-435f-860c-abd4211b5f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:27.952080 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aac694-5376-435f-860c-abd4211b5f41 None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:27.952832 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aac694-5376-435f-860c-abd4211b5f41 None admin] RBAC: Adding query filter params (type=image) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:27.953146 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aac694-5376-435f-860c-abd4211b5f41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:27.954517 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aac694-5376-435f-860c-abd4211b5f41 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:27.959220 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 498/1016] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:07:27 2022] GET /identity/v3/services?type=image => generated 122 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:07:28.022686 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96940ed8-a65d-4da3-ab4a-3a0aea4b421b None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:28.065687 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc6c8b-54ec-469b-9f2a-c22802e15e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:28.066715 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc6c8b-54ec-469b-9f2a-c22802e15e0e None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:28.067176 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc6c8b-54ec-469b-9f2a-c22802e15e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:28.067602 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc6c8b-54ec-469b-9f2a-c22802e15e0e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:28.068570 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cc6c8b-54ec-469b-9f2a-c22802e15e0e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:28.070346 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cc6c8b-54ec-469b-9f2a-c22802e15e0e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:28.188907 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 519/1017] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Jul 18 06:07:27 2022] POST /identity/v3/endpoints => generated 334 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:07:29.177448 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9002c1-232a-47fd-af01-9002d5f596dd None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:29.177989 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9002c1-232a-47fd-af01-9002d5f596dd None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:29.177989 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9002c1-232a-47fd-af01-9002d5f596dd None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:29.179666 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 499/1018] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:07:29.185177 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d26b67-c6d6-4e7e-adba-dbe1e2fbcc75 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:29.185667 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d26b67-c6d6-4e7e-adba-dbe1e2fbcc75 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:29.185747 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d26b67-c6d6-4e7e-adba-dbe1e2fbcc75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:29.202639 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-41d26b67-c6d6-4e7e-adba-dbe1e2fbcc75 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:29.203819 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-41d26b67-c6d6-4e7e-adba-dbe1e2fbcc75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:29.272263 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:29.280155 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 520/1019] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:29 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:07:29.304886 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9002c1-232a-47fd-af01-9002d5f596dd None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:29.416294 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9002c1-232a-47fd-af01-9002d5f596dd None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:29.451143 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a988c4cf-930a-49a1-b35c-84a376727590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:29.451849 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a988c4cf-930a-49a1-b35c-84a376727590 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:29.452132 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a988c4cf-930a-49a1-b35c-84a376727590 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:29.452434 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a988c4cf-930a-49a1-b35c-84a376727590 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:29.453590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a988c4cf-930a-49a1-b35c-84a376727590 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:29.455020 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a988c4cf-930a-49a1-b35c-84a376727590 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:29.456836 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 500/1020] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:07:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:29.468442 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d26b67-c6d6-4e7e-adba-dbe1e2fbcc75 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:29.569249 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d26b67-c6d6-4e7e-adba-dbe1e2fbcc75 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:29.604366 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3341634-68cb-4994-8c9f-acb880fb017d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:29.605074 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3341634-68cb-4994-8c9f-acb880fb017d None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:29.605363 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3341634-68cb-4994-8c9f-acb880fb017d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:29.605646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3341634-68cb-4994-8c9f-acb880fb017d None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:29.606376 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3341634-68cb-4994-8c9f-acb880fb017d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:29.607804 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3341634-68cb-4994-8c9f-acb880fb017d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:29.714276 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 521/1021] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Jul 18 06:07:29 2022] POST /identity/v3/users => generated 285 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:07:30.022335 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c27dd5-88a7-4619-9e73-e9f5a6401f94 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:30.022802 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c27dd5-88a7-4619-9e73-e9f5a6401f94 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:30.023048 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c27dd5-88a7-4619-9e73-e9f5a6401f94 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:30.024532 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 501/1022] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:07:30.029800 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc4a4b-0e79-4d0d-a30a-12dbdef7c9d0 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:30.030098 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc4a4b-0e79-4d0d-a30a-12dbdef7c9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:30.030379 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc4a4b-0e79-4d0d-a30a-12dbdef7c9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:30.047678 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-3edc4a4b-0e79-4d0d-a30a-12dbdef7c9d0 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:30.048931 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-3edc4a4b-0e79-4d0d-a30a-12dbdef7c9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:30.106492 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:30.125016 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 522/1023] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:30 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:07:30.148318 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c27dd5-88a7-4619-9e73-e9f5a6401f94 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:30.182683 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:30.257736 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c27dd5-88a7-4619-9e73-e9f5a6401f94 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:30.295191 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c4ec8-b449-4cfb-9b68-495e43de68ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:30.296101 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c4ec8-b449-4cfb-9b68-495e43de68ed None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:30.296575 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c4ec8-b449-4cfb-9b68-495e43de68ed None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:30.296993 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c4ec8-b449-4cfb-9b68-495e43de68ed None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:30.297865 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10c4ec8-b449-4cfb-9b68-495e43de68ed None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:30.299476 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10c4ec8-b449-4cfb-9b68-495e43de68ed None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:30.342996 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d08567-5e53-4ec4-9020-69cb16a5de32 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:30.343542 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d08567-5e53-4ec4-9020-69cb16a5de32 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:30.343542 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d08567-5e53-4ec4-9020-69cb16a5de32 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:30.344966 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 523/1024] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:07:30.349881 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45efd4de-3761-4924-9217-248afe79e530 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:30.350166 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45efd4de-3761-4924-9217-248afe79e530 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:30.352647 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45efd4de-3761-4924-9217-248afe79e530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:30.368979 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-45efd4de-3761-4924-9217-248afe79e530 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:30.370242 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-45efd4de-3761-4924-9217-248afe79e530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:30.377547 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 502/1025] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Jul 18 06:07:30 2022] POST /identity/v3/services => generated 242 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:07:30.460321 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 524/1026] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:30 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 06:07:30.484056 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c4ec8-b449-4cfb-9b68-495e43de68ed None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:30.593058 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:30.602853 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c4ec8-b449-4cfb-9b68-495e43de68ed None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:30.638805 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d018f-3c9a-4fe4-9ffd-4ce863e3bfd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:30.639625 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d018f-3c9a-4fe4-9ffd-4ce863e3bfd7 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:30.639913 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d018f-3c9a-4fe4-9ffd-4ce863e3bfd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:30.640209 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d018f-3c9a-4fe4-9ffd-4ce863e3bfd7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:30.652813 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d018f-3c9a-4fe4-9ffd-4ce863e3bfd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:07:30.662023 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-932d018f-3c9a-4fe4-9ffd-4ce863e3bfd7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 06:07:30.662023 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:30.662023 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:30.662023 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:30.662023 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:30.662023 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:30.662023 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:30.662023 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:30.662023 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:30.662023 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:30.662023 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:30.662023 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:30.662023 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:07:30.662023 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:07:30.662023 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:07:30.662023 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:07:30.662968 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:07:30.662968 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:07:30.662968 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:07:30.662968 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:07:30.662968 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:30.662968 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:30.662968 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:30.662968 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:30.662968 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:30.662968 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:30.662968 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:30.662968 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:30.662968 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:30.662968 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:30.662968 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:30.662968 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:30.663886 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:30.663886 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:30.663886 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:30.663886 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:30.663886 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:07:30.663886 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:07:30.663886 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:07:30.663886 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:07:30.663886 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:07:30.663886 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:07:30.663886 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 06:07:30.663886 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:30.664557 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 503/1027] 10.222.0.40 () {56 vars in 1120 bytes} [Mon Jul 18 06:07:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:07:30.670798 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45efd4de-3761-4924-9217-248afe79e530 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:30.784076 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45efd4de-3761-4924-9217-248afe79e530 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:30.796357 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:30.820525 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78730321-80f4-4876-b186-638b52e6ddb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:30.821231 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78730321-80f4-4876-b186-638b52e6ddb9 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:30.821516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78730321-80f4-4876-b186-638b52e6ddb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:30.821795 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78730321-80f4-4876-b186-638b52e6ddb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:30.822578 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78730321-80f4-4876-b186-638b52e6ddb9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:30.822889 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78730321-80f4-4876-b186-638b52e6ddb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:30.824271 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78730321-80f4-4876-b186-638b52e6ddb9 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:30.837715 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 525/1028] 10.222.0.40 () {56 vars in 1115 bytes} [Mon Jul 18 06:07:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:07:30.839197 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d018f-3c9a-4fe4-9ffd-4ce863e3bfd7 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:30.946901 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d018f-3c9a-4fe4-9ffd-4ce863e3bfd7 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:30.982425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9298191-96d0-4894-ba7d-29f22ca7f8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:30.983223 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9298191-96d0-4894-ba7d-29f22ca7f8b0 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:30.983506 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9298191-96d0-4894-ba7d-29f22ca7f8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:30.983782 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9298191-96d0-4894-ba7d-29f22ca7f8b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:30.996028 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9298191-96d0-4894-ba7d-29f22ca7f8b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:30.997794 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9298191-96d0-4894-ba7d-29f22ca7f8b0 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:31.016075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-c9298191-96d0-4894-ba7d-29f22ca7f8b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:31.016075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:31.016075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:31.016075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:31.016075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:31.016075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:31.016075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:31.016075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:31.016075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:31.016075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:31.016075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:31.016075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:31.016075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:07:31.016075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:07:31.016075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:07:31.017029 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:07:31.017029 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:31.017029 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:31.017029 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:31.017029 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:31.017029 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:31.017029 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:31.017029 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:31.017029 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:31.017029 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:31.017029 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:31.017029 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:31.017029 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:31.017029 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:31.017029 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:31.017029 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:31.017029 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:31.017973 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:07:31.017973 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:07:31.017973 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:31.017973 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:31.017973 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 504/1029] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:07:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:07:31.023800 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78730321-80f4-4876-b186-638b52e6ddb9 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:31.133901 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78730321-80f4-4876-b186-638b52e6ddb9 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:31.169820 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a1484-522b-40d2-a607-cf91624e2dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:31.170497 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a1484-522b-40d2-a607-cf91624e2dcf None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:31.170784 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a1484-522b-40d2-a607-cf91624e2dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:31.171064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a1484-522b-40d2-a607-cf91624e2dcf None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:31.171799 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4a1484-522b-40d2-a607-cf91624e2dcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:31.172111 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4a1484-522b-40d2-a607-cf91624e2dcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:31.173542 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4a1484-522b-40d2-a607-cf91624e2dcf None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:31.186537 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 526/1030] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:07:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:31.192276 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9298191-96d0-4894-ba7d-29f22ca7f8b0 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:31.196305 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:31.297253 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9298191-96d0-4894-ba7d-29f22ca7f8b0 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:31.331860 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e67f1-e315-4033-8031-59d5a971df47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:31.332691 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e67f1-e315-4033-8031-59d5a971df47 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:31.332979 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e67f1-e315-4033-8031-59d5a971df47 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:31.333254 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e67f1-e315-4033-8031-59d5a971df47 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:31.341556 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798e67f1-e315-4033-8031-59d5a971df47 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:31.342978 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798e67f1-e315-4033-8031-59d5a971df47 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:31.357692 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-798e67f1-e315-4033-8031-59d5a971df47 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 06:07:31.357692 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:31.357692 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:31.357692 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:31.357692 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:31.357692 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:31.357692 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:31.357692 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:31.357692 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:31.357692 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:31.357692 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:31.357692 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:31.357692 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 06:07:31.357692 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 06:07:31.357692 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 06:07:31.357692 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 06:07:31.358761 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:31.358761 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:31.358761 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 06:07:31.358761 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:07:31.358761 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 06:07:31.358761 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:07:31.358761 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:31.358761 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:31.358761 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:31.358761 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:31.358761 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:31.358761 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:31.358761 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:31.358761 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:31.358761 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:31.358761 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:31.358761 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:31.358761 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:31.359858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:31.359858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:31.359858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 06:07:31.359858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 06:07:31.359858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 06:07:31.359858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 06:07:31.359858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 06:07:31.359858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 06:07:31.359858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 06:07:31.359858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:31.363701 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 505/1031] 10.222.0.40 () {56 vars in 1093 bytes} [Mon Jul 18 06:07:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:31.364708 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a1484-522b-40d2-a607-cf91624e2dcf None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:31.402619 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:31.479275 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a1484-522b-40d2-a607-cf91624e2dcf None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:31.521690 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a8e988-d623-479e-b602-99d1763f7c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:31.522401 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8e988-d623-479e-b602-99d1763f7c9a None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:31.522782 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8e988-d623-479e-b602-99d1763f7c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:31.523064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8e988-d623-479e-b602-99d1763f7c9a None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:31.523921 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a8e988-d623-479e-b602-99d1763f7c9a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:31.524235 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a8e988-d623-479e-b602-99d1763f7c9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:31.525710 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a8e988-d623-479e-b602-99d1763f7c9a None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:31.544364 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 527/1032] 10.222.0.40 () {56 vars in 1133 bytes} [Mon Jul 18 06:07:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:07:31.550068 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e67f1-e315-4033-8031-59d5a971df47 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:31.619438 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:31.660770 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e67f1-e315-4033-8031-59d5a971df47 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:31.699193 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf25e3-cd64-4277-bce6-1907fa5f3bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:31.699462 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf25e3-cd64-4277-bce6-1907fa5f3bae None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:31.699599 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf25e3-cd64-4277-bce6-1907fa5f3bae None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:31.699871 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf25e3-cd64-4277-bce6-1907fa5f3bae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:31.770495 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaf25e3-cd64-4277-bce6-1907fa5f3bae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:31.771930 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaf25e3-cd64-4277-bce6-1907fa5f3bae None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:31.830097 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f608916-cd16-4ae1-b3c5-19b166b0121c None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:31.830393 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f608916-cd16-4ae1-b3c5-19b166b0121c None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:31.830588 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f608916-cd16-4ae1-b3c5-19b166b0121c None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:31.832080 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 528/1033] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:07:31.833934 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:31.837028 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a45a1-b251-4939-af48-f23b68a6d15b None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:31.841075 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a45a1-b251-4939-af48-f23b68a6d15b None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:31.841075 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a45a1-b251-4939-af48-f23b68a6d15b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:31.854412 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-556a45a1-b251-4939-af48-f23b68a6d15b None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:31.855669 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-556a45a1-b251-4939-af48-f23b68a6d15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:31.882080 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 506/1034] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:07:31 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 06:07:31.883254 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf25e3-cd64-4277-bce6-1907fa5f3bae None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:31.945396 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 529/1035] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:31 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 06:07:31.967487 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a45a1-b251-4939-af48-f23b68a6d15b None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:31.989901 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:32.001186 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf25e3-cd64-4277-bce6-1907fa5f3bae None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:32.039580 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa7c6a-0c5e-4aae-b2e9-8a656607b8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:32.040379 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa7c6a-0c5e-4aae-b2e9-8a656607b8a4 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:32.040705 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa7c6a-0c5e-4aae-b2e9-8a656607b8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:32.040985 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa7c6a-0c5e-4aae-b2e9-8a656607b8a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:32.052089 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa7c6a-0c5e-4aae-b2e9-8a656607b8a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:32.053619 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa7c6a-0c5e-4aae-b2e9-8a656607b8a4 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:32.072503 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-58aa7c6a-0c5e-4aae-b2e9-8a656607b8a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:32.072503 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:32.072503 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:32.072503 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:32.072503 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:32.072503 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:32.072503 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:32.072503 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:32.072503 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:32.072503 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:32.072503 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:32.072503 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:32.072503 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:07:32.072503 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:07:32.072503 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:07:32.073499 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:07:32.073499 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:32.073499 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:32.073499 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:32.073499 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:32.073499 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:32.073499 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:32.073499 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:32.073499 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:32.073499 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:32.073499 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:32.073499 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:32.073499 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:32.073499 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:32.073499 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:32.073499 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:32.073499 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:32.074442 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:07:32.074442 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:07:32.074442 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:32.074442 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:32.074442 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 507/1036] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:07:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:07:32.079270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa7c6a-0c5e-4aae-b2e9-8a656607b8a4 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:32.083658 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a45a1-b251-4939-af48-f23b68a6d15b None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:32.098628 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:32.122041 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1d0638-8959-4e76-adcf-6b4459e54bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:32.123172 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d0638-8959-4e76-adcf-6b4459e54bf8 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:32.123664 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d0638-8959-4e76-adcf-6b4459e54bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:32.124136 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d0638-8959-4e76-adcf-6b4459e54bf8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:32.136382 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1d0638-8959-4e76-adcf-6b4459e54bf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:07:32.144751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-5e1d0638-8959-4e76-adcf-6b4459e54bf8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:07:32.144751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:32.144751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:32.144751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:32.144751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:32.144751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:32.144751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:32.144751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:32.144751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:32.144751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:32.144751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:32.144751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:32.144751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:07:32.144751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:07:32.144751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:07:32.144751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:07:32.145776 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:07:32.145776 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:07:32.145776 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:07:32.145776 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:07:32.145776 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:32.145776 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:32.145776 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:32.145776 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:32.145776 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:32.145776 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:32.145776 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:32.145776 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:32.145776 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:32.145776 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:32.145776 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:32.145776 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:32.146646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:32.146646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:32.146646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:32.146646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:32.146646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:07:32.146646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:07:32.146646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:07:32.146646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:07:32.146646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:07:32.146646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:07:32.146646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:07:32.146646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:32.149452 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 530/1037] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 18 06:07:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:07:32.155326 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1d0638-8959-4e76-adcf-6b4459e54bf8 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:32.188681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa7c6a-0c5e-4aae-b2e9-8a656607b8a4 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:32.228715 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea181c-05bf-469b-88d8-db32374faeb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:32.229495 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea181c-05bf-469b-88d8-db32374faeb7 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:32.229826 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea181c-05bf-469b-88d8-db32374faeb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:32.230105 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea181c-05bf-469b-88d8-db32374faeb7 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:32.230895 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ea181c-05bf-469b-88d8-db32374faeb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:32.231209 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ea181c-05bf-469b-88d8-db32374faeb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:32.232607 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ea181c-05bf-469b-88d8-db32374faeb7 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:32.249852 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 508/1038] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:07:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:07:32.250060 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:32.250887 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea181c-05bf-469b-88d8-db32374faeb7 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:32.270765 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1d0638-8959-4e76-adcf-6b4459e54bf8 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:32.312887 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890da40c-e0f0-4e8c-9d74-921c9f45573f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:32.313565 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890da40c-e0f0-4e8c-9d74-921c9f45573f None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:32.313855 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890da40c-e0f0-4e8c-9d74-921c9f45573f None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:32.314135 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890da40c-e0f0-4e8c-9d74-921c9f45573f None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:32.314890 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890da40c-e0f0-4e8c-9d74-921c9f45573f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:32.315200 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890da40c-e0f0-4e8c-9d74-921c9f45573f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:32.316619 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890da40c-e0f0-4e8c-9d74-921c9f45573f None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:32.326823 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 531/1039] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:07:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:32.332798 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890da40c-e0f0-4e8c-9d74-921c9f45573f None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:32.347724 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:32.371511 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea181c-05bf-469b-88d8-db32374faeb7 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:32.408188 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b08377-7c59-4ace-b269-16263641ae67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:32.409136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b08377-7c59-4ace-b269-16263641ae67 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:32.409423 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b08377-7c59-4ace-b269-16263641ae67 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:32.409706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b08377-7c59-4ace-b269-16263641ae67 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:32.417435 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b08377-7c59-4ace-b269-16263641ae67 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:32.418872 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b08377-7c59-4ace-b269-16263641ae67 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:32.434717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-56b08377-7c59-4ace-b269-16263641ae67 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:07:32.434717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:32.434717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:32.434717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:32.434717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:32.434717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:32.434717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:32.434717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:32.434717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:32.434717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:32.434717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:32.434717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:32.434717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 06:07:32.434717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 06:07:32.434717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 06:07:32.435717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 06:07:32.435717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:32.435717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:32.435717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:32.435717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:32.435717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:32.435717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:32.435717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:32.435717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:32.435717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:32.435717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:32.435717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:32.435717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:32.435717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:32.435717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:32.435717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:32.435717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:32.436682 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 06:07:32.436682 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 06:07:32.436682 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 06:07:32.436682 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 06:07:32.436682 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 06:07:32.436682 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 06:07:32.436682 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:07:32.436682 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:32.436682 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 509/1040] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:07:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:32.441726 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b08377-7c59-4ace-b269-16263641ae67 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:32.447908 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890da40c-e0f0-4e8c-9d74-921c9f45573f None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:32.486026 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe2e34a-eadd-4d85-8f1b-6b1c7e0ff8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:32.486910 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2e34a-eadd-4d85-8f1b-6b1c7e0ff8d6 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:32.487193 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2e34a-eadd-4d85-8f1b-6b1c7e0ff8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:32.487469 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2e34a-eadd-4d85-8f1b-6b1c7e0ff8d6 None admin] PATH_INFO: `/v3/users/cd6cf8b57e984a3bba55f01fa0a3bf6b` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:32.500346 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:32.502923 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe2e34a-eadd-4d85-8f1b-6b1c7e0ff8d6 None admin] RBAC: Authorizing `identity:get_user(user_id=cd6cf8b57e984a3bba55f01fa0a3bf6b)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:32.504424 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe2e34a-eadd-4d85-8f1b-6b1c7e0ff8d6 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:32.506296 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 532/1041] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:07:32 2022] GET /identity/v3/users/cd6cf8b57e984a3bba55f01fa0a3bf6b => generated 285 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:07:32.511987 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe2e34a-eadd-4d85-8f1b-6b1c7e0ff8d6 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:32.559035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b08377-7c59-4ace-b269-16263641ae67 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:32.593831 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc05fd21-536b-44c1-a4b3-313739f8eebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:32.594459 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05fd21-536b-44c1-a4b3-313739f8eebc None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:32.594740 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05fd21-536b-44c1-a4b3-313739f8eebc None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:32.595010 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05fd21-536b-44c1-a4b3-313739f8eebc None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:32.595796 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc05fd21-536b-44c1-a4b3-313739f8eebc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:32.596104 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc05fd21-536b-44c1-a4b3-313739f8eebc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:32.597564 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc05fd21-536b-44c1-a4b3-313739f8eebc None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:32.600067 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:32.609704 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 510/1042] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 18 06:07:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:07:32.615049 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863f10f-c1a5-4706-ae4c-4addb7b15542 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:32.615049 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863f10f-c1a5-4706-ae4c-4addb7b15542 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:32.615049 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863f10f-c1a5-4706-ae4c-4addb7b15542 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:32.615049 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 511/1043] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:07:32.617842 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2863f10f-c1a5-4706-ae4c-4addb7b15542 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:32.622289 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe2e34a-eadd-4d85-8f1b-6b1c7e0ff8d6 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:32.657919 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f9e0a1-131b-48d7-8240-8f583c3b8d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:32.658574 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9e0a1-131b-48d7-8240-8f583c3b8d2c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:32.658857 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9e0a1-131b-48d7-8240-8f583c3b8d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:32.659135 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9e0a1-131b-48d7-8240-8f583c3b8d2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:32.725312 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2863f10f-c1a5-4706-ae4c-4addb7b15542 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:32.731574 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:32.733932 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f9e0a1-131b-48d7-8240-8f583c3b8d2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:32.735417 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f9e0a1-131b-48d7-8240-8f583c3b8d2c None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:32.761633 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3d2926-a177-4f87-a786-83803d817da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:32.762348 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d2926-a177-4f87-a786-83803d817da3 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:32.762640 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d2926-a177-4f87-a786-83803d817da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:32.762932 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d2926-a177-4f87-a786-83803d817da3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:32.763795 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3d2926-a177-4f87-a786-83803d817da3 None admin] RBAC: Adding query filter params (role.id=34f769e98e1e449cb45aa4b31836393a, scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a, user.id=f92d948810ea4241810473506b654f7a) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:32.764108 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3d2926-a177-4f87-a786-83803d817da3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:32.765576 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3d2926-a177-4f87-a786-83803d817da3 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:32.771238 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 512/1044] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:07:32 2022] GET /identity/v3/role_assignments?user.id=f92d948810ea4241810473506b654f7a&scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a&role.id=34f769e98e1e449cb45aa4b31836393a => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:32.772974 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37efc792-86a7-4bcd-b756-0d14b2a4393a None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:32.773387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37efc792-86a7-4bcd-b756-0d14b2a4393a None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:32.773659 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37efc792-86a7-4bcd-b756-0d14b2a4393a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:32.803633 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-37efc792-86a7-4bcd-b756-0d14b2a4393a None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:32.804931 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-37efc792-86a7-4bcd-b756-0d14b2a4393a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:32.858960 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 533/1045] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:07:32 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 06:07:32.860297 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f9e0a1-131b-48d7-8240-8f583c3b8d2c None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:32.873509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:32.886411 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 513/1046] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:32 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:07:32.910477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37efc792-86a7-4bcd-b756-0d14b2a4393a None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:32.977382 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:32.980615 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f9e0a1-131b-48d7-8240-8f583c3b8d2c None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:33.017941 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b742b-6761-491e-b827-267e1776e0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:33.018818 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b742b-6761-491e-b827-267e1776e0eb None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:33.019105 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b742b-6761-491e-b827-267e1776e0eb None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:33.019383 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b742b-6761-491e-b827-267e1776e0eb None admin] PATH_INFO: `/v3/projects/72e5dfa90eb34aabb7d344eff121a0e5` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:33.020645 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5b742b-6761-491e-b827-267e1776e0eb None admin] RBAC: Authorizing `identity:get_project(project_id=72e5dfa90eb34aabb7d344eff121a0e5)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:33.022111 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5b742b-6761-491e-b827-267e1776e0eb None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:33.025041 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 534/1047] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:07:32 2022] GET /identity/v3/projects/72e5dfa90eb34aabb7d344eff121a0e5 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:33.030732 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b742b-6761-491e-b827-267e1776e0eb None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:33.034861 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37efc792-86a7-4bcd-b756-0d14b2a4393a None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:33.074939 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c877c-1580-4c8b-9f2a-5d0bf65f2374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:33.075769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c877c-1580-4c8b-9f2a-5d0bf65f2374 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:33.076057 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c877c-1580-4c8b-9f2a-5d0bf65f2374 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:33.076338 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c877c-1580-4c8b-9f2a-5d0bf65f2374 None admin] PATH_INFO: `/v3/services/image` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:33.077034 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c877c-1580-4c8b-9f2a-5d0bf65f2374 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:33.078599 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c877c-1580-4c8b-9f2a-5d0bf65f2374 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:33.092509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-d26c877c-1580-4c8b-9f2a-5d0bf65f2374 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 06:07:33.092509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:33.092509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 06:07:33.092509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 06:07:33.092509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 06:07:33.092509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 06:07:33.092509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 06:07:33.092509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:07:33.092509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 06:07:33.092509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:33.092509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 06:07:33.092509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:33.092509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:33.092509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:33.092509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:33.092509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:33.092509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:33.093620 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:33.093620 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:33.093620 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:33.093620 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:33.093620 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:33.093620 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:33.093620 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 06:07:33.093620 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 06:07:33.093620 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 06:07:33.093620 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 06:07:33.093620 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:33.093620 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:33.093620 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:33.093620 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:33.093620 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:33.093620 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:33.094503 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:33.094503 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:33.094503 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:33.094503 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:33.094503 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:33.094503 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:33.094503 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:33.094503 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:33.094503 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:33.094503 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:33.094503 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 06:07:33.094503 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:07:33.094503 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 06:07:33.094503 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:33.094503 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 514/1048] 10.222.0.40 () {56 vars in 1105 bytes} [Mon Jul 18 06:07:32 2022] GET /identity/v3/services/image => generated 86 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:07:33.099822 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c877c-1580-4c8b-9f2a-5d0bf65f2374 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:33.150439 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b742b-6761-491e-b827-267e1776e0eb None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:33.171686 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:33.193606 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e307dd-4535-44b1-9982-aa28925f7e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:33.195077 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e307dd-4535-44b1-9982-aa28925f7e71 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:33.195374 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e307dd-4535-44b1-9982-aa28925f7e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:33.195646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e307dd-4535-44b1-9982-aa28925f7e71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:33.196804 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e307dd-4535-44b1-9982-aa28925f7e71 None admin] RBAC: Adding query filter params (role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc, scope.project.id=72e5dfa90eb34aabb7d344eff121a0e5, user.id=cd6cf8b57e984a3bba55f01fa0a3bf6b) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:33.196804 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e307dd-4535-44b1-9982-aa28925f7e71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:33.198397 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e307dd-4535-44b1-9982-aa28925f7e71 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:33.204458 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 535/1049] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:07:33 2022] GET /identity/v3/role_assignments?user.id=cd6cf8b57e984a3bba55f01fa0a3bf6b&scope.project.id=72e5dfa90eb34aabb7d344eff121a0e5&role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:33.234688 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c877c-1580-4c8b-9f2a-5d0bf65f2374 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:33.275065 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d67f3a-7717-4c06-8e97-ba567136bb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:33.276461 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d67f3a-7717-4c06-8e97-ba567136bb86 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:33.276880 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d67f3a-7717-4c06-8e97-ba567136bb86 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:33.277298 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d67f3a-7717-4c06-8e97-ba567136bb86 None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:33.278214 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d67f3a-7717-4c06-8e97-ba567136bb86 None admin] RBAC: Adding query filter params (name=image) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:33.278744 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d67f3a-7717-4c06-8e97-ba567136bb86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:33.280350 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d67f3a-7717-4c06-8e97-ba567136bb86 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:33.286673 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 515/1050] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:07:33 2022] GET /identity/v3/services?name=image => generated 122 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:07:33.292681 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e307dd-4535-44b1-9982-aa28925f7e71 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:33.328881 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:33.404648 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e307dd-4535-44b1-9982-aa28925f7e71 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:33.442807 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c91937-2c27-4f6e-b677-ec85ce7ecee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:33.443530 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c91937-2c27-4f6e-b677-ec85ce7ecee8 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:33.443817 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c91937-2c27-4f6e-b677-ec85ce7ecee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:33.444101 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c91937-2c27-4f6e-b677-ec85ce7ecee8 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:33.445677 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c91937-2c27-4f6e-b677-ec85ce7ecee8 None admin] RBAC: Adding query filter params (type=image) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:33.445990 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c91937-2c27-4f6e-b677-ec85ce7ecee8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:33.447376 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c91937-2c27-4f6e-b677-ec85ce7ecee8 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:33.456574 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 536/1051] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:07:33 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:33.458498 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d67f3a-7717-4c06-8e97-ba567136bb86 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:33.566335 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d67f3a-7717-4c06-8e97-ba567136bb86 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:33.585509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:33.602734 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff521267-97a7-4ab1-907e-4b8ed12633c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:33.603396 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff521267-97a7-4ab1-907e-4b8ed12633c6 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:33.603689 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff521267-97a7-4ab1-907e-4b8ed12633c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:33.604974 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff521267-97a7-4ab1-907e-4b8ed12633c6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:33.604974 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff521267-97a7-4ab1-907e-4b8ed12633c6 None admin] RBAC: Adding query filter params (interface=public, service_id=fb8ec31c5ecf4f6c97b55efa6023d0ef, region_id=RegionOne) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:33.605141 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff521267-97a7-4ab1-907e-4b8ed12633c6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:33.606473 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff521267-97a7-4ab1-907e-4b8ed12633c6 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:33.616623 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 516/1052] 10.222.0.40 () {56 vars in 1251 bytes} [Mon Jul 18 06:07:33 2022] GET /identity/v3/endpoints?service_id=fb8ec31c5ecf4f6c97b55efa6023d0ef&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:33.618189 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c91937-2c27-4f6e-b677-ec85ce7ecee8 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:33.719622 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:33.725385 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c91937-2c27-4f6e-b677-ec85ce7ecee8 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:33.761225 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a015fb1-13fb-487c-8750-ace643ac8dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:33.761895 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a015fb1-13fb-487c-8750-ace643ac8dde None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:33.762181 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a015fb1-13fb-487c-8750-ace643ac8dde None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:33.762461 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a015fb1-13fb-487c-8750-ace643ac8dde None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:33.763165 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a015fb1-13fb-487c-8750-ace643ac8dde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:33.764572 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a015fb1-13fb-487c-8750-ace643ac8dde None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:33.771438 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 537/1053] 10.222.0.40 () {56 vars in 1087 bytes} [Mon Jul 18 06:07:33 2022] GET /identity/v3/services => generated 1957 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:07:34.820440 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa5d40-828f-48e2-b5e7-df8c45bfd175 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:34.820978 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa5d40-828f-48e2-b5e7-df8c45bfd175 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:34.820978 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa5d40-828f-48e2-b5e7-df8c45bfd175 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:34.822418 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 517/1054] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:07:34.828101 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac594a-c73b-43f3-a98d-1e03e1ed4aa2 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:34.828576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac594a-c73b-43f3-a98d-1e03e1ed4aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:34.828747 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac594a-c73b-43f3-a98d-1e03e1ed4aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:34.845476 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-15ac594a-c73b-43f3-a98d-1e03e1ed4aa2 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:34.846664 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-15ac594a-c73b-43f3-a98d-1e03e1ed4aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:34.920702 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 538/1055] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:34 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:07:34.943497 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa5d40-828f-48e2-b5e7-df8c45bfd175 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:35.051302 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa5d40-828f-48e2-b5e7-df8c45bfd175 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:35.085287 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a10ccb9-333d-4c4f-9cd8-7d3fed99dcaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:35.086350 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10ccb9-333d-4c4f-9cd8-7d3fed99dcaf None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:35.086634 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10ccb9-333d-4c4f-9cd8-7d3fed99dcaf None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:35.086909 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10ccb9-333d-4c4f-9cd8-7d3fed99dcaf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:35.097447 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a10ccb9-333d-4c4f-9cd8-7d3fed99dcaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:07:35.106207 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-9a10ccb9-333d-4c4f-9cd8-7d3fed99dcaf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 06:07:35.106207 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:35.106207 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:35.106207 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:35.106207 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:35.106207 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:35.106207 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:35.106207 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:35.106207 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:35.106207 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:35.106207 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:35.106207 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:35.106207 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:07:35.106207 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:07:35.106207 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:07:35.106207 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:07:35.107150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:07:35.107150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:07:35.107150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:07:35.107150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:07:35.107150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:35.107150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:35.107150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:35.107150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:35.107150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:35.107150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:35.107150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:35.107150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:35.107150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:35.107150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:35.107150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:35.107150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:35.108022 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:35.108022 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:35.108022 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:35.108022 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:35.108022 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:07:35.108022 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:07:35.108022 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:07:35.108022 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:07:35.108022 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:07:35.108022 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:07:35.108022 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 06:07:35.108022 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:35.108022 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 518/1056] 10.222.0.40 () {56 vars in 1120 bytes} [Mon Jul 18 06:07:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:35.113516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac594a-c73b-43f3-a98d-1e03e1ed4aa2 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:35.206791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:35.217295 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac594a-c73b-43f3-a98d-1e03e1ed4aa2 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:35.253630 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455d2647-474b-4e54-9307-8411bf747205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:35.254308 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d2647-474b-4e54-9307-8411bf747205 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:35.254585 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d2647-474b-4e54-9307-8411bf747205 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:35.254924 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d2647-474b-4e54-9307-8411bf747205 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:35.255607 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455d2647-474b-4e54-9307-8411bf747205 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:35.255917 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455d2647-474b-4e54-9307-8411bf747205 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:35.257370 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455d2647-474b-4e54-9307-8411bf747205 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:35.270230 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 539/1057] 10.222.0.40 () {56 vars in 1115 bytes} [Mon Jul 18 06:07:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:35.271658 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a10ccb9-333d-4c4f-9cd8-7d3fed99dcaf None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:35.281001 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da6e22-f4b5-4c04-b56d-37f823acbd3f None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:35.281001 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da6e22-f4b5-4c04-b56d-37f823acbd3f None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:35.281283 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da6e22-f4b5-4c04-b56d-37f823acbd3f None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:35.282878 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 540/1058] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:07:35.292050 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110fffa3-7441-436c-9295-868542300355 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:35.292350 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110fffa3-7441-436c-9295-868542300355 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:35.292651 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110fffa3-7441-436c-9295-868542300355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:35.313993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-110fffa3-7441-436c-9295-868542300355 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:35.315193 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-110fffa3-7441-436c-9295-868542300355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:35.320293 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:35.379740 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a10ccb9-333d-4c4f-9cd8-7d3fed99dcaf None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:35.402191 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 541/1059] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:35 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 06:07:35.417572 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b399088f-2af4-4e33-b606-9a7ff301e526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:35.418372 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399088f-2af4-4e33-b606-9a7ff301e526 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:35.418659 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399088f-2af4-4e33-b606-9a7ff301e526 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:35.418943 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399088f-2af4-4e33-b606-9a7ff301e526 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:35.426511 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110fffa3-7441-436c-9295-868542300355 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:35.431570 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b399088f-2af4-4e33-b606-9a7ff301e526 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:35.433154 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b399088f-2af4-4e33-b606-9a7ff301e526 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:35.452174 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-b399088f-2af4-4e33-b606-9a7ff301e526 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:35.452174 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:35.452174 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:35.452174 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:35.452174 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:35.452174 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:35.452174 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:35.452174 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:35.452174 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:35.452174 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:35.452174 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:35.452174 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:35.452174 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:07:35.452174 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:07:35.452174 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:07:35.453297 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:07:35.453297 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:35.453297 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:35.453297 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:35.453297 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:35.453297 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:35.453297 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:35.453297 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:35.453297 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:35.453297 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:35.453297 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:35.453297 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:35.453297 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:35.453297 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:35.453297 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:35.453297 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:35.453297 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:35.454208 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:07:35.454208 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:07:35.454208 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:35.454208 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:35.454208 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 519/1060] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:07:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:35.459103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b399088f-2af4-4e33-b606-9a7ff301e526 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:35.525387 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:35.545901 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110fffa3-7441-436c-9295-868542300355 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:35.561218 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:35.568137 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b399088f-2af4-4e33-b606-9a7ff301e526 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:35.584205 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0703a89-450a-4e23-88fe-c3d9acf4c230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:35.585827 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0703a89-450a-4e23-88fe-c3d9acf4c230 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:35.586133 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0703a89-450a-4e23-88fe-c3d9acf4c230 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:35.586407 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0703a89-450a-4e23-88fe-c3d9acf4c230 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:35.597018 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0703a89-450a-4e23-88fe-c3d9acf4c230 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:07:35.604862 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-f0703a89-450a-4e23-88fe-c3d9acf4c230 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:07:35.604862 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:35.604862 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:35.604862 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:35.604862 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:35.604862 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:35.604862 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:35.604862 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:35.604862 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:35.604862 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:35.604862 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:35.604862 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:35.604862 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:07:35.604862 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:07:35.604862 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:07:35.604862 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:07:35.605810 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:07:35.605810 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:07:35.605810 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:07:35.605810 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:07:35.605810 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:35.605810 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:35.605810 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:35.605810 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:35.605810 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:35.605810 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:35.605810 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:35.605810 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:35.605810 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:35.605810 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:35.605810 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:35.605810 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:35.606664 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:35.606664 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:35.606664 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:35.606664 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:35.606664 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:07:35.606664 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:07:35.606664 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:07:35.606664 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:07:35.606664 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:07:35.606664 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:07:35.606664 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:07:35.606664 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:35.607318 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bbb5d0-9413-4002-8aa3-771535778117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:35.607318 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbb5d0-9413-4002-8aa3-771535778117 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:35.607318 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbb5d0-9413-4002-8aa3-771535778117 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:35.607318 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbb5d0-9413-4002-8aa3-771535778117 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:35.607606 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bbb5d0-9413-4002-8aa3-771535778117 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:35.608228 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bbb5d0-9413-4002-8aa3-771535778117 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:35.609176 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bbb5d0-9413-4002-8aa3-771535778117 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:35.611254 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 542/1061] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 18 06:07:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:07:35.612105 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0703a89-450a-4e23-88fe-c3d9acf4c230 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:35.621629 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 520/1062] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:07:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:07:35.627039 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bbb5d0-9413-4002-8aa3-771535778117 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:35.726015 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0703a89-450a-4e23-88fe-c3d9acf4c230 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:35.748242 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bbb5d0-9413-4002-8aa3-771535778117 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:35.763538 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e70e4a-4469-4734-a1cc-3d1aa94ab614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:35.764217 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e70e4a-4469-4734-a1cc-3d1aa94ab614 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:35.764528 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e70e4a-4469-4734-a1cc-3d1aa94ab614 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:35.764811 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e70e4a-4469-4734-a1cc-3d1aa94ab614 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:35.765559 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e70e4a-4469-4734-a1cc-3d1aa94ab614 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:35.765862 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e70e4a-4469-4734-a1cc-3d1aa94ab614 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:35.767233 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e70e4a-4469-4734-a1cc-3d1aa94ab614 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:35.776583 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 543/1063] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:07:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:35.782146 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e70e4a-4469-4734-a1cc-3d1aa94ab614 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:35.786175 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:35.787444 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce15bd-38de-4b4f-ad91-3c811cb78977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:35.792086 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce15bd-38de-4b4f-ad91-3c811cb78977 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:35.792086 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce15bd-38de-4b4f-ad91-3c811cb78977 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:35.792086 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce15bd-38de-4b4f-ad91-3c811cb78977 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:35.796952 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce15bd-38de-4b4f-ad91-3c811cb78977 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:35.798903 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce15bd-38de-4b4f-ad91-3c811cb78977 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:35.814154 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-5bce15bd-38de-4b4f-ad91-3c811cb78977 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 06:07:35.814154 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:35.814154 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:35.814154 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:35.814154 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:35.814154 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:35.814154 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:35.814154 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:35.814154 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:35.814154 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:35.814154 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:35.814154 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:35.814154 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 06:07:35.814154 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 06:07:35.814154 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 06:07:35.814154 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 06:07:35.815270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:35.815270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:35.815270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 06:07:35.815270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:07:35.815270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 06:07:35.815270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:07:35.815270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:35.815270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:35.815270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:35.815270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:35.815270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:35.815270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:35.815270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:35.815270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:35.815270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:35.815270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:35.815270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:35.815270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:35.816238 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:35.816238 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:35.816238 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 06:07:35.816238 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 06:07:35.816238 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 06:07:35.816238 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 06:07:35.816238 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 06:07:35.816238 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 06:07:35.816238 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 06:07:35.816238 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:35.816238 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 521/1064] 10.222.0.40 () {56 vars in 1093 bytes} [Mon Jul 18 06:07:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:07:35.822129 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce15bd-38de-4b4f-ad91-3c811cb78977 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:35.840320 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:35.894444 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e70e4a-4469-4734-a1cc-3d1aa94ab614 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:35.927296 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce15bd-38de-4b4f-ad91-3c811cb78977 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:35.929766 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c625a3a-9692-4ead-8fd8-5cb9d3d57b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:35.930652 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c625a3a-9692-4ead-8fd8-5cb9d3d57b3b None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:35.930941 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c625a3a-9692-4ead-8fd8-5cb9d3d57b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:35.931226 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c625a3a-9692-4ead-8fd8-5cb9d3d57b3b None admin] PATH_INFO: `/v3/users/cd6cf8b57e984a3bba55f01fa0a3bf6b` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:35.932490 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c625a3a-9692-4ead-8fd8-5cb9d3d57b3b None admin] RBAC: Authorizing `identity:get_user(user_id=cd6cf8b57e984a3bba55f01fa0a3bf6b)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:35.933940 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c625a3a-9692-4ead-8fd8-5cb9d3d57b3b None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:35.935591 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 544/1065] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:07:35 2022] GET /identity/v3/users/cd6cf8b57e984a3bba55f01fa0a3bf6b => generated 285 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:07:35.938191 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5331f73-d5c5-4dbb-8eb2-293f8755362e None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:35.938701 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5331f73-d5c5-4dbb-8eb2-293f8755362e None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:35.939036 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5331f73-d5c5-4dbb-8eb2-293f8755362e None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:35.940656 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 545/1066] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:07:35.941371 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5331f73-d5c5-4dbb-8eb2-293f8755362e None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:35.966323 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e977d4-b919-41b4-823b-31eeffe8a87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:35.966998 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e977d4-b919-41b4-823b-31eeffe8a87c None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:35.967283 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e977d4-b919-41b4-823b-31eeffe8a87c None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:35.967561 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e977d4-b919-41b4-823b-31eeffe8a87c None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:35.968400 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e977d4-b919-41b4-823b-31eeffe8a87c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:35.968741 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e977d4-b919-41b4-823b-31eeffe8a87c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:35.970170 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e977d4-b919-41b4-823b-31eeffe8a87c None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:35.989003 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 522/1067] 10.222.0.40 () {56 vars in 1133 bytes} [Mon Jul 18 06:07:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:07:35.991808 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119ef9a-e9e3-485e-aae7-dcb607b5e28b None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:35.991808 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119ef9a-e9e3-485e-aae7-dcb607b5e28b None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:35.991808 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119ef9a-e9e3-485e-aae7-dcb607b5e28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:36.006468 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-b119ef9a-e9e3-485e-aae7-dcb607b5e28b None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:36.007788 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b119ef9a-e9e3-485e-aae7-dcb607b5e28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:36.020963 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:36.056514 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5331f73-d5c5-4dbb-8eb2-293f8755362e None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:36.082045 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 523/1068] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:35 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:07:36.082984 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b119ef9a-e9e3-485e-aae7-dcb607b5e28b None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:36.086274 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:36.126342 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e3e3a0-398e-4c38-95bb-e6f4d6f092ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:36.127225 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3e3a0-398e-4c38-95bb-e6f4d6f092ed None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:36.127508 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3e3a0-398e-4c38-95bb-e6f4d6f092ed None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:36.127786 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3e3a0-398e-4c38-95bb-e6f4d6f092ed None admin] PATH_INFO: `/v3/projects/72e5dfa90eb34aabb7d344eff121a0e5` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:36.129062 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e3e3a0-398e-4c38-95bb-e6f4d6f092ed None admin] RBAC: Authorizing `identity:get_project(project_id=72e5dfa90eb34aabb7d344eff121a0e5)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:36.130537 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e3e3a0-398e-4c38-95bb-e6f4d6f092ed None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:36.132850 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 546/1069] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:07:35 2022] GET /identity/v3/projects/72e5dfa90eb34aabb7d344eff121a0e5 => generated 311 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:36.133639 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e3e3a0-398e-4c38-95bb-e6f4d6f092ed None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:36.396073 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b119ef9a-e9e3-485e-aae7-dcb607b5e28b None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:36.427965 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e3e3a0-398e-4c38-95bb-e6f4d6f092ed None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:36.430709 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55655d3-b1fc-463e-86ce-3d9a8edc65a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:36.431655 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55655d3-b1fc-463e-86ce-3d9a8edc65a5 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:36.431938 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55655d3-b1fc-463e-86ce-3d9a8edc65a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:36.432214 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55655d3-b1fc-463e-86ce-3d9a8edc65a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:36.443455 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55655d3-b1fc-463e-86ce-3d9a8edc65a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:36.445453 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55655d3-b1fc-463e-86ce-3d9a8edc65a5 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:36.463165 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba96ad4-2f0a-4973-b016-4600441a0287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:36.463590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-f55655d3-b1fc-463e-86ce-3d9a8edc65a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:36.463590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:36.463590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:36.463590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:36.463590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:36.463590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:36.463590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:36.463590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:36.463590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:36.463590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:36.463590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:36.463590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:36.463590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:07:36.463590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:07:36.463590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:07:36.464774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:07:36.464774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:36.464774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:36.464774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:36.464774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:36.464774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:36.464774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:36.464774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:36.464774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:36.464774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:36.464774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:36.464774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:36.464774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:36.464774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:36.464774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:36.464774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:36.464774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:36.465772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:07:36.465772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:07:36.465772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:36.465772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:36.466010 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba96ad4-2f0a-4973-b016-4600441a0287 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:36.466010 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba96ad4-2f0a-4973-b016-4600441a0287 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:36.466010 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba96ad4-2f0a-4973-b016-4600441a0287 None admin] PATH_INFO: `/v3/services/image` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:36.466189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 524/1070] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:07:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:36.466298 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba96ad4-2f0a-4973-b016-4600441a0287 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:36.466377 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55655d3-b1fc-463e-86ce-3d9a8edc65a5 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:36.466552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba96ad4-2f0a-4973-b016-4600441a0287 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:36.480445 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-aba96ad4-2f0a-4973-b016-4600441a0287 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 06:07:36.480445 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:36.480445 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 06:07:36.480445 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 06:07:36.480445 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 06:07:36.480445 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 06:07:36.480445 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 06:07:36.480445 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:07:36.480445 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 06:07:36.480445 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:36.480445 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 06:07:36.480445 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:36.480445 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:36.480445 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:36.480445 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:36.480445 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:36.480445 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:36.481750 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:36.481750 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:36.481750 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:36.481750 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:36.481750 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:36.481750 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:36.481750 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 06:07:36.481750 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 06:07:36.481750 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 06:07:36.481750 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 06:07:36.481750 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:36.481750 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:36.481750 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:36.481750 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:36.481750 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:36.481750 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:36.482606 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:36.482606 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:36.482606 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:36.482606 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:36.482606 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:36.482606 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:36.482606 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:36.482606 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:36.482606 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:36.482606 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:36.482606 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 06:07:36.482606 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:07:36.482606 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 06:07:36.482606 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:36.482606 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 547/1071] 10.222.0.40 () {56 vars in 1105 bytes} [Mon Jul 18 06:07:36 2022] GET /identity/v3/services/image => generated 86 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:36.483412 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba96ad4-2f0a-4973-b016-4600441a0287 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:36.490064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:36.527634 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:36.574465 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55655d3-b1fc-463e-86ce-3d9a8edc65a5 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:36.590382 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba96ad4-2f0a-4973-b016-4600441a0287 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:36.609039 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87260e77-27c4-40d9-987c-68c6d3b5478d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:36.609930 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87260e77-27c4-40d9-987c-68c6d3b5478d None admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:36.610218 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87260e77-27c4-40d9-987c-68c6d3b5478d None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:36.610494 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87260e77-27c4-40d9-987c-68c6d3b5478d None admin] PATH_INFO: `/v3/projects/72e5dfa90eb34aabb7d344eff121a0e5/users/cd6cf8b57e984a3bba55f01fa0a3bf6b/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:36.612474 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87260e77-27c4-40d9-987c-68c6d3b5478d None admin] RBAC: Authorizing `identity:create_grant(project_id=72e5dfa90eb34aabb7d344eff121a0e5, user_id=cd6cf8b57e984a3bba55f01fa0a3bf6b, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:36.613902 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87260e77-27c4-40d9-987c-68c6d3b5478d None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:36.624665 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f4ff9-1f43-441f-bb64-fbf48dfa3b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:36.625331 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f4ff9-1f43-441f-bb64-fbf48dfa3b18 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:36.625620 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f4ff9-1f43-441f-bb64-fbf48dfa3b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:36.625949 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f4ff9-1f43-441f-bb64-fbf48dfa3b18 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:36.626677 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0f4ff9-1f43-441f-bb64-fbf48dfa3b18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:36.627015 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0f4ff9-1f43-441f-bb64-fbf48dfa3b18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:36.628384 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0f4ff9-1f43-441f-bb64-fbf48dfa3b18 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:36.640966 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 548/1072] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:07:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:36.641762 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f4ff9-1f43-441f-bb64-fbf48dfa3b18 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:36.684319 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 525/1073] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 18 06:07:36 2022] PUT /identity/v3/projects/72e5dfa90eb34aabb7d344eff121a0e5/users/cd6cf8b57e984a3bba55f01fa0a3bf6b/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:07:36.685182 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87260e77-27c4-40d9-987c-68c6d3b5478d None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:36.717016 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:36.761694 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f4ff9-1f43-441f-bb64-fbf48dfa3b18 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:36.805526 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d464252-863d-40aa-a438-f7285addedb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:36.806228 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d464252-863d-40aa-a438-f7285addedb9 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:36.806548 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d464252-863d-40aa-a438-f7285addedb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:36.807062 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d464252-863d-40aa-a438-f7285addedb9 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:36.807810 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d464252-863d-40aa-a438-f7285addedb9 None admin] RBAC: Adding query filter params (name=image) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:36.808250 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d464252-863d-40aa-a438-f7285addedb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:36.809852 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d464252-863d-40aa-a438-f7285addedb9 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:36.822668 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 549/1074] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:07:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:36.823933 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d464252-863d-40aa-a438-f7285addedb9 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:36.833884 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87260e77-27c4-40d9-987c-68c6d3b5478d None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:36.854961 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:36.876965 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0eb46-7f3e-40b6-b622-c11c6ca21392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:36.877767 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0eb46-7f3e-40b6-b622-c11c6ca21392 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:36.878056 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0eb46-7f3e-40b6-b622-c11c6ca21392 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:36.878366 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0eb46-7f3e-40b6-b622-c11c6ca21392 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:36.886112 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e0eb46-7f3e-40b6-b622-c11c6ca21392 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:36.887556 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e0eb46-7f3e-40b6-b622-c11c6ca21392 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:36.902907 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-82e0eb46-7f3e-40b6-b622-c11c6ca21392 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:07:36.902907 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:36.902907 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:36.902907 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:36.902907 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:36.902907 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:36.902907 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:36.902907 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:36.902907 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:36.902907 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:36.902907 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:36.902907 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:36.902907 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 06:07:36.902907 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 06:07:36.902907 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 06:07:36.904025 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 06:07:36.904025 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:36.904025 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:36.904025 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:36.904025 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:36.904025 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:36.904025 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:36.904025 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:36.904025 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:36.904025 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:36.904025 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:36.904025 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:36.904025 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:36.904025 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:36.904025 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:36.904025 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:36.904025 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:36.905075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 06:07:36.905075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 06:07:36.905075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 06:07:36.905075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 06:07:36.905075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 06:07:36.905075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 06:07:36.905075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:07:36.905075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:36.905075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 526/1075] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:07:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:36.909972 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0eb46-7f3e-40b6-b622-c11c6ca21392 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:36.936524 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d464252-863d-40aa-a438-f7285addedb9 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:36.965406 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:36.973846 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b63bfd-39e9-43f4-abf8-bf95c10a842c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:36.974516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b63bfd-39e9-43f4-abf8-bf95c10a842c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:36.974798 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b63bfd-39e9-43f4-abf8-bf95c10a842c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:36.975072 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b63bfd-39e9-43f4-abf8-bf95c10a842c None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:36.976505 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b63bfd-39e9-43f4-abf8-bf95c10a842c None admin] RBAC: Adding query filter params (type=image) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:36.976843 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b63bfd-39e9-43f4-abf8-bf95c10a842c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:36.978192 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b63bfd-39e9-43f4-abf8-bf95c10a842c None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:36.983696 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 550/1076] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:07:36 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:07:36.989213 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b63bfd-39e9-43f4-abf8-bf95c10a842c None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:37.016446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0eb46-7f3e-40b6-b622-c11c6ca21392 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:37.058502 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa02f7-627d-4cd4-87fd-a952bcc1f7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:37.059315 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa02f7-627d-4cd4-87fd-a952bcc1f7f9 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:37.059517 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa02f7-627d-4cd4-87fd-a952bcc1f7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:37.059799 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa02f7-627d-4cd4-87fd-a952bcc1f7f9 None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:37.060689 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aa02f7-627d-4cd4-87fd-a952bcc1f7f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:37.061002 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aa02f7-627d-4cd4-87fd-a952bcc1f7f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:37.062562 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aa02f7-627d-4cd4-87fd-a952bcc1f7f9 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:37.079631 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 527/1077] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 18 06:07:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:37.080968 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa02f7-627d-4cd4-87fd-a952bcc1f7f9 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:37.106568 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b63bfd-39e9-43f4-abf8-bf95c10a842c None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:37.121501 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:37.143909 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e29ab-f1ad-4451-9eb8-f5a79a475d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:37.144602 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e29ab-f1ad-4451-9eb8-f5a79a475d01 None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:37.144890 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e29ab-f1ad-4451-9eb8-f5a79a475d01 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:37.145171 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e29ab-f1ad-4451-9eb8-f5a79a475d01 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:37.146090 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61e29ab-f1ad-4451-9eb8-f5a79a475d01 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:37.147684 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61e29ab-f1ad-4451-9eb8-f5a79a475d01 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:37.197783 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa02f7-627d-4cd4-87fd-a952bcc1f7f9 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:37.230476 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 551/1078] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Jul 18 06:07:36 2022] POST /identity/v3/endpoints => generated 327 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:07:37.251908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5619551a-e724-4f41-8b0f-4fb0caaced82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:37.252979 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619551a-e724-4f41-8b0f-4fb0caaced82 None admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:37.253271 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619551a-e724-4f41-8b0f-4fb0caaced82 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:37.253556 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619551a-e724-4f41-8b0f-4fb0caaced82 None admin] PATH_INFO: `/v3/projects/8ed50a8a3e184bde9269d7b0d9573d4a/users/f92d948810ea4241810473506b654f7a/roles/34f769e98e1e449cb45aa4b31836393a` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:37.256020 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5619551a-e724-4f41-8b0f-4fb0caaced82 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ed50a8a3e184bde9269d7b0d9573d4a, user_id=f92d948810ea4241810473506b654f7a, role_id=34f769e98e1e449cb45aa4b31836393a)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:37.257872 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5619551a-e724-4f41-8b0f-4fb0caaced82 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:37.348680 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 528/1079] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 18 06:07:37 2022] PUT /identity/v3/projects/8ed50a8a3e184bde9269d7b0d9573d4a/users/f92d948810ea4241810473506b654f7a/roles/34f769e98e1e449cb45aa4b31836393a => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:07:38.743305 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63568df2-f271-421e-8abf-bf8a6d696a3b None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:38.743860 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63568df2-f271-421e-8abf-bf8a6d696a3b None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:38.743860 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63568df2-f271-421e-8abf-bf8a6d696a3b None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:38.745276 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 552/1080] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:07:38.750369 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5cf07e-6371-4003-8408-69d06d118d20 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:38.750778 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5cf07e-6371-4003-8408-69d06d118d20 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:38.750962 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5cf07e-6371-4003-8408-69d06d118d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:38.768370 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-dd5cf07e-6371-4003-8408-69d06d118d20 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:38.769581 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-dd5cf07e-6371-4003-8408-69d06d118d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:38.837202 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 529/1081] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:07:38.860826 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63568df2-f271-421e-8abf-bf8a6d696a3b None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:38.927988 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:38.995463 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63568df2-f271-421e-8abf-bf8a6d696a3b None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:39.036380 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4003b-530a-4127-88b4-7b421798d706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:39.037588 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4003b-530a-4127-88b4-7b421798d706 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:39.038692 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4003b-530a-4127-88b4-7b421798d706 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:39.039198 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4003b-530a-4127-88b4-7b421798d706 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:39.051268 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c4003b-530a-4127-88b4-7b421798d706 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:07:39.059372 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-a6c4003b-530a-4127-88b4-7b421798d706 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:07:39.059372 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:39.059372 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:39.059372 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:39.059372 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:39.059372 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:39.059372 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:39.059372 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:39.059372 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:39.059372 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:39.059372 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:39.059372 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:39.059372 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:07:39.059372 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:07:39.059372 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:07:39.059372 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:07:39.060454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:07:39.060454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:07:39.060454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:07:39.060454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:07:39.060454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:39.060454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:39.060454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:39.060454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:39.060454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:39.060454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:39.060454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:39.060454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:39.060454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:39.060454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:39.060454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:39.060454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:39.061319 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:39.061319 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:39.061319 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:39.061319 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:39.061319 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:07:39.061319 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:07:39.061319 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:07:39.061319 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:07:39.061319 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:07:39.061319 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:07:39.061319 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:07:39.061319 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:39.064565 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 553/1082] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 18 06:07:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:39.067624 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5cf07e-6371-4003-8408-69d06d118d20 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:39.097747 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:39.183159 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5cf07e-6371-4003-8408-69d06d118d20 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:39.221012 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25c5d3-b92f-4445-9f85-39e935a71480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:39.221683 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25c5d3-b92f-4445-9f85-39e935a71480 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:39.221964 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25c5d3-b92f-4445-9f85-39e935a71480 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:39.222240 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25c5d3-b92f-4445-9f85-39e935a71480 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:39.222989 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b25c5d3-b92f-4445-9f85-39e935a71480 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:39.223293 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b25c5d3-b92f-4445-9f85-39e935a71480 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:39.224711 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b25c5d3-b92f-4445-9f85-39e935a71480 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:39.239991 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 530/1083] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:07:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:39.241511 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4003b-530a-4127-88b4-7b421798d706 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:39.342309 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bab5a5-6cf1-480b-a96f-3ab3d260e671 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:39.342612 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bab5a5-6cf1-480b-a96f-3ab3d260e671 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:39.342799 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bab5a5-6cf1-480b-a96f-3ab3d260e671 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:39.344206 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 531/1084] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:07:39.349552 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31f4a1f-7f3c-4db2-bdf6-6748a5c0240e None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:39.349873 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31f4a1f-7f3c-4db2-bdf6-6748a5c0240e None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:39.350714 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31f4a1f-7f3c-4db2-bdf6-6748a5c0240e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:39.362878 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4003b-530a-4127-88b4-7b421798d706 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:39.368801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-b31f4a1f-7f3c-4db2-bdf6-6748a5c0240e None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:39.370008 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b31f4a1f-7f3c-4db2-bdf6-6748a5c0240e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:39.372805 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:39.401098 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2fc5e-8b3a-4635-b75a-021289e17c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:39.401988 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2fc5e-8b3a-4635-b75a-021289e17c1f None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:39.402278 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2fc5e-8b3a-4635-b75a-021289e17c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:39.402561 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2fc5e-8b3a-4635-b75a-021289e17c1f None admin] PATH_INFO: `/v3/users/cd6cf8b57e984a3bba55f01fa0a3bf6b` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:39.403885 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd2fc5e-8b3a-4635-b75a-021289e17c1f None admin] RBAC: Authorizing `identity:get_user(user_id=cd6cf8b57e984a3bba55f01fa0a3bf6b)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:39.405396 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd2fc5e-8b3a-4635-b75a-021289e17c1f None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:39.412201 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 554/1085] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:07:39 2022] GET /identity/v3/users/cd6cf8b57e984a3bba55f01fa0a3bf6b => generated 285 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:39.415992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2fc5e-8b3a-4635-b75a-021289e17c1f None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:39.447521 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 532/1086] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 06:07:39.448892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eededdfc-f5fc-4bcd-acbe-649a1bd16bfc None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:39.449190 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eededdfc-f5fc-4bcd-acbe-649a1bd16bfc None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:39.449471 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eededdfc-f5fc-4bcd-acbe-649a1bd16bfc None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:39.452604 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 533/1087] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:07:39.457861 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788871d-d450-4036-9d75-b06233ebc552 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:39.458219 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788871d-d450-4036-9d75-b06233ebc552 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:39.459053 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788871d-d450-4036-9d75-b06233ebc552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:39.480509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-d788871d-d450-4036-9d75-b06233ebc552 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:39.481722 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-d788871d-d450-4036-9d75-b06233ebc552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:39.489528 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:39.523430 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2fc5e-8b3a-4635-b75a-021289e17c1f None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:39.551878 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 534/1088] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 06:07:39.552820 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788871d-d450-4036-9d75-b06233ebc552 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:39.560830 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3cb3f1-bab6-44ce-bcd9-e1a51201756a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:39.561493 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cb3f1-bab6-44ce-bcd9-e1a51201756a None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:39.561781 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cb3f1-bab6-44ce-bcd9-e1a51201756a None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:39.562070 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cb3f1-bab6-44ce-bcd9-e1a51201756a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:39.615439 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:39.644185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3cb3f1-bab6-44ce-bcd9-e1a51201756a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:39.646707 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3cb3f1-bab6-44ce-bcd9-e1a51201756a None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:39.667425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788871d-d450-4036-9d75-b06233ebc552 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:39.711493 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be19a9-7e49-4625-94ab-f5192e0712d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:39.712160 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be19a9-7e49-4625-94ab-f5192e0712d4 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:39.712476 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be19a9-7e49-4625-94ab-f5192e0712d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:39.712765 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be19a9-7e49-4625-94ab-f5192e0712d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:39.728757 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:39.760278 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 555/1089] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:07:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 06:07:39.761176 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3cb3f1-bab6-44ce-bcd9-e1a51201756a None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:39.795914 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7be19a9-7e49-4625-94ab-f5192e0712d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:39.797418 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7be19a9-7e49-4625-94ab-f5192e0712d4 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:39.843970 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:39.868646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3cb3f1-bab6-44ce-bcd9-e1a51201756a None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:39.906921 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882e8b56-52be-4aa5-b9cc-35ed3e29e1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:39.907733 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e8b56-52be-4aa5-b9cc-35ed3e29e1b1 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:39.908009 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e8b56-52be-4aa5-b9cc-35ed3e29e1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:39.908280 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e8b56-52be-4aa5-b9cc-35ed3e29e1b1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:39.919813 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882e8b56-52be-4aa5-b9cc-35ed3e29e1b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:07:39.920981 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 535/1090] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:07:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 06:07:39.921832 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be19a9-7e49-4625-94ab-f5192e0712d4 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:39.928139 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-882e8b56-52be-4aa5-b9cc-35ed3e29e1b1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 06:07:39.928139 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:39.928139 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:39.928139 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:39.928139 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:39.928139 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:39.928139 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:39.928139 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:39.928139 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:39.928139 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:39.928139 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:39.928139 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:39.928139 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:07:39.928139 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:07:39.928139 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:07:39.928139 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:07:39.930049 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:07:39.930049 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:07:39.930049 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:07:39.930049 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:07:39.930049 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:39.930049 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:39.930049 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:39.930049 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:39.930049 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:39.930049 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:39.930049 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:39.930049 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:39.930049 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:39.930049 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:39.930049 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:39.930049 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:39.930944 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:39.930944 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:39.930944 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:39.930944 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:39.930944 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:07:39.930944 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:07:39.930944 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:07:39.930944 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:07:39.930944 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:07:39.930944 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:07:39.930944 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 06:07:39.930944 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:39.930944 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 556/1091] 10.222.0.40 () {56 vars in 1120 bytes} [Mon Jul 18 06:07:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:39.931680 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882e8b56-52be-4aa5-b9cc-35ed3e29e1b1 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:39.968029 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:40.029490 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be19a9-7e49-4625-94ab-f5192e0712d4 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:40.036190 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882e8b56-52be-4aa5-b9cc-35ed3e29e1b1 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:40.066049 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a7b3b0-3f31-44bd-8dd0-f6b80f481b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:40.066907 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7b3b0-3f31-44bd-8dd0-f6b80f481b05 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:40.067196 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7b3b0-3f31-44bd-8dd0-f6b80f481b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:40.067470 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7b3b0-3f31-44bd-8dd0-f6b80f481b05 None admin] PATH_INFO: `/v3/projects/72e5dfa90eb34aabb7d344eff121a0e5` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:40.068696 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a7b3b0-3f31-44bd-8dd0-f6b80f481b05 None admin] RBAC: Authorizing `identity:get_project(project_id=72e5dfa90eb34aabb7d344eff121a0e5)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:40.070195 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a7b3b0-3f31-44bd-8dd0-f6b80f481b05 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:40.072095 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 536/1092] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:07:39 2022] GET /identity/v3/projects/72e5dfa90eb34aabb7d344eff121a0e5 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:40.073037 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a7b3b0-3f31-44bd-8dd0-f6b80f481b05 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:40.073722 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c69342c-e459-47b8-963d-9d08644d9f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:40.074656 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69342c-e459-47b8-963d-9d08644d9f1f None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:40.075080 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69342c-e459-47b8-963d-9d08644d9f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:40.075498 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69342c-e459-47b8-963d-9d08644d9f1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:40.079477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:40.089293 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c69342c-e459-47b8-963d-9d08644d9f1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:40.090794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c69342c-e459-47b8-963d-9d08644d9f1f None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:40.109574 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-4c69342c-e459-47b8-963d-9d08644d9f1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:40.109574 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:40.109574 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:40.109574 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:40.109574 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:40.109574 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:40.109574 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:40.109574 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:40.109574 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:40.109574 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:40.109574 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:40.109574 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:40.109574 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:07:40.109574 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:07:40.109574 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:07:40.110764 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:07:40.110764 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:40.110764 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:40.110764 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:40.110764 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:40.110764 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:40.110764 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:40.110764 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:40.110764 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:40.110764 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:40.110764 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:40.110764 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:40.110764 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:40.110764 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:40.110764 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:40.110764 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:40.110764 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:40.111705 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:07:40.111705 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:07:40.111705 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:40.111705 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:40.111705 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 557/1093] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:07:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:40.111705 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c69342c-e459-47b8-963d-9d08644d9f1f None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:40.187700 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a7b3b0-3f31-44bd-8dd0-f6b80f481b05 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:40.198840 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:40.217054 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c69342c-e459-47b8-963d-9d08644d9f1f None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:40.222636 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce9f35e-503b-4f36-9bd8-ac9f46ac2e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:40.223579 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9f35e-503b-4f36-9bd8-ac9f46ac2e79 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:40.223897 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9f35e-503b-4f36-9bd8-ac9f46ac2e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:40.224206 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9f35e-503b-4f36-9bd8-ac9f46ac2e79 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:40.225016 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce9f35e-503b-4f36-9bd8-ac9f46ac2e79 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:40.225860 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce9f35e-503b-4f36-9bd8-ac9f46ac2e79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:40.227442 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce9f35e-503b-4f36-9bd8-ac9f46ac2e79 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:40.235914 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 537/1094] 10.222.0.40 () {56 vars in 1115 bytes} [Mon Jul 18 06:07:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:40.236800 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce9f35e-503b-4f36-9bd8-ac9f46ac2e79 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:40.253144 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f53c6-ef50-42f7-99e0-9ce11f0a02f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:40.253809 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f53c6-ef50-42f7-99e0-9ce11f0a02f1 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:40.254095 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f53c6-ef50-42f7-99e0-9ce11f0a02f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:40.254374 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f53c6-ef50-42f7-99e0-9ce11f0a02f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:40.255212 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f53c6-ef50-42f7-99e0-9ce11f0a02f1 None admin] RBAC: Adding query filter params (role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc, scope.project.id=72e5dfa90eb34aabb7d344eff121a0e5, user.id=cd6cf8b57e984a3bba55f01fa0a3bf6b) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:40.255526 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f53c6-ef50-42f7-99e0-9ce11f0a02f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:40.257032 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f53c6-ef50-42f7-99e0-9ce11f0a02f1 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:40.262972 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 558/1095] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:07:40 2022] GET /identity/v3/role_assignments?user.id=cd6cf8b57e984a3bba55f01fa0a3bf6b&scope.project.id=72e5dfa90eb34aabb7d344eff121a0e5&role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:40.263792 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f53c6-ef50-42f7-99e0-9ce11f0a02f1 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:40.342909 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:40.356522 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce9f35e-503b-4f36-9bd8-ac9f46ac2e79 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:40.394491 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c821ff8-0c8f-493b-b8b8-4b9a5f681555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:40.395160 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c821ff8-0c8f-493b-b8b8-4b9a5f681555 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:40.395452 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c821ff8-0c8f-493b-b8b8-4b9a5f681555 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:40.395730 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c821ff8-0c8f-493b-b8b8-4b9a5f681555 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:40.396519 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c821ff8-0c8f-493b-b8b8-4b9a5f681555 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:40.396836 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c821ff8-0c8f-493b-b8b8-4b9a5f681555 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:40.399109 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c821ff8-0c8f-493b-b8b8-4b9a5f681555 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:40.400860 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f53c6-ef50-42f7-99e0-9ce11f0a02f1 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:40.412608 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 538/1096] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:07:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:40.451418 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e508e3-dae1-4952-923e-0c44d859589d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:40.452166 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e508e3-dae1-4952-923e-0c44d859589d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:40.452512 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e508e3-dae1-4952-923e-0c44d859589d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:40.452798 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e508e3-dae1-4952-923e-0c44d859589d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:40.466962 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e508e3-dae1-4952-923e-0c44d859589d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:40.468575 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e508e3-dae1-4952-923e-0c44d859589d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:40.488746 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-f1e508e3-dae1-4952-923e-0c44d859589d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:40.488746 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:40.488746 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:40.488746 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:40.488746 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:40.488746 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:40.488746 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:40.488746 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:40.488746 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:40.488746 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:40.488746 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:40.488746 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:40.488746 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:07:40.488746 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:07:40.488746 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:07:40.489734 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:07:40.489734 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:40.489734 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:40.489734 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:40.489734 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:40.489734 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:40.489734 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:40.489734 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:40.489734 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:40.489734 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:40.489734 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:40.489734 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:40.489734 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:40.489734 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:40.489734 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:40.489734 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:40.489734 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:40.490647 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:07:40.490647 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:07:40.490647 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:40.490647 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:40.490647 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 559/1097] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:07:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:40.497350 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c821ff8-0c8f-493b-b8b8-4b9a5f681555 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:40.536380 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:40.624687 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c821ff8-0c8f-493b-b8b8-4b9a5f681555 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:40.667876 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ff242-6e31-4055-86c1-7b9cf0099cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:40.669096 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ff242-6e31-4055-86c1-7b9cf0099cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:40.669416 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ff242-6e31-4055-86c1-7b9cf0099cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:40.669700 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ff242-6e31-4055-86c1-7b9cf0099cb5 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:40.670448 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352ff242-6e31-4055-86c1-7b9cf0099cb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:40.670788 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352ff242-6e31-4055-86c1-7b9cf0099cb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:40.672274 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352ff242-6e31-4055-86c1-7b9cf0099cb5 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:40.688480 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 539/1098] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:07:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:07:40.693059 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e508e3-dae1-4952-923e-0c44d859589d None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:40.798231 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e508e3-dae1-4952-923e-0c44d859589d None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:40.800511 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:40.832635 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1acb25-39c6-4970-8e6d-1a0ea4ded55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:40.833435 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1acb25-39c6-4970-8e6d-1a0ea4ded55d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:40.833725 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1acb25-39c6-4970-8e6d-1a0ea4ded55d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:40.834008 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1acb25-39c6-4970-8e6d-1a0ea4ded55d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:40.842440 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1acb25-39c6-4970-8e6d-1a0ea4ded55d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:40.843907 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1acb25-39c6-4970-8e6d-1a0ea4ded55d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:40.859847 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-4f1acb25-39c6-4970-8e6d-1a0ea4ded55d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 06:07:40.859847 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:40.859847 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:40.859847 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:40.859847 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:40.859847 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:40.859847 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:40.859847 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:40.859847 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:40.859847 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:40.859847 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:40.859847 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:40.859847 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 06:07:40.859847 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 06:07:40.859847 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 06:07:40.859847 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 06:07:40.861806 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:40.861806 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:40.861806 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 06:07:40.861806 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:07:40.861806 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 06:07:40.861806 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:07:40.861806 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:40.861806 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:40.861806 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:40.861806 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:40.861806 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:40.861806 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:40.861806 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:40.861806 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:40.861806 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:40.861806 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:40.861806 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:40.861806 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:40.863675 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:40.863675 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:40.863675 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 06:07:40.863675 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 06:07:40.863675 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 06:07:40.863675 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 06:07:40.863675 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 06:07:40.863675 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 06:07:40.863675 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 06:07:40.863675 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:40.863675 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 560/1099] 10.222.0.40 () {56 vars in 1093 bytes} [Mon Jul 18 06:07:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:07:40.867502 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ff242-6e31-4055-86c1-7b9cf0099cb5 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:40.960114 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:40.968791 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ff242-6e31-4055-86c1-7b9cf0099cb5 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:41.003636 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7807c74-aa5b-4045-ba40-2b4efd65816a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:41.004541 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7807c74-aa5b-4045-ba40-2b4efd65816a None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:41.005004 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7807c74-aa5b-4045-ba40-2b4efd65816a None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:41.005653 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7807c74-aa5b-4045-ba40-2b4efd65816a None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:41.006499 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7807c74-aa5b-4045-ba40-2b4efd65816a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:41.006809 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7807c74-aa5b-4045-ba40-2b4efd65816a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:41.008235 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7807c74-aa5b-4045-ba40-2b4efd65816a None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:41.026576 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 540/1100] 10.222.0.40 () {56 vars in 1133 bytes} [Mon Jul 18 06:07:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:41.032030 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1acb25-39c6-4970-8e6d-1a0ea4ded55d None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:41.148620 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1acb25-39c6-4970-8e6d-1a0ea4ded55d None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:41.184307 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea09629-e9a7-4550-87b7-4c92a52ad5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:41.185001 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea09629-e9a7-4550-87b7-4c92a52ad5be None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:41.185292 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea09629-e9a7-4550-87b7-4c92a52ad5be None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:41.185574 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea09629-e9a7-4550-87b7-4c92a52ad5be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:41.224050 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:41.262714 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea09629-e9a7-4550-87b7-4c92a52ad5be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:41.264187 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea09629-e9a7-4550-87b7-4c92a52ad5be None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:41.377342 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 561/1101] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:07:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 06:07:41.383738 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7807c74-aa5b-4045-ba40-2b4efd65816a None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:41.489310 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7807c74-aa5b-4045-ba40-2b4efd65816a None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:41.527328 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73f940-5d97-45de-825b-32c2cef2fd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:41.528121 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73f940-5d97-45de-825b-32c2cef2fd27 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:41.528401 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73f940-5d97-45de-825b-32c2cef2fd27 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:41.528701 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73f940-5d97-45de-825b-32c2cef2fd27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:41.539963 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f73f940-5d97-45de-825b-32c2cef2fd27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:41.541434 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f73f940-5d97-45de-825b-32c2cef2fd27 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:41.562114 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-6f73f940-5d97-45de-825b-32c2cef2fd27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:41.562114 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:41.562114 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:41.562114 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:41.562114 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:41.562114 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:41.562114 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:41.562114 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:41.562114 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:41.562114 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:41.562114 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:41.562114 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:41.562114 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:07:41.562114 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:07:41.562114 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:07:41.563315 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:07:41.563315 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:41.563315 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:41.563315 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:41.563315 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:41.563315 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:41.563315 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:41.563315 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:41.563315 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:41.563315 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:41.563315 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:41.563315 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:41.563315 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:41.563315 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:41.563315 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:41.563315 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:41.563315 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:41.564221 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:07:41.564221 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:07:41.564221 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:07:41.564221 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:41.564221 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 541/1102] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:07:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:07:41.569767 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea09629-e9a7-4550-87b7-4c92a52ad5be None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:41.634477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:41.674223 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea09629-e9a7-4550-87b7-4c92a52ad5be None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:41.711486 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b6dd0-0ef9-4d72-b09a-d00440b7199a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:41.712471 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b6dd0-0ef9-4d72-b09a-d00440b7199a None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:41.712761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b6dd0-0ef9-4d72-b09a-d00440b7199a None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:41.713036 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b6dd0-0ef9-4d72-b09a-d00440b7199a None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:41.713770 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b6dd0-0ef9-4d72-b09a-d00440b7199a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:41.714077 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b6dd0-0ef9-4d72-b09a-d00440b7199a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:41.715458 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b6dd0-0ef9-4d72-b09a-d00440b7199a None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:41.728031 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 562/1103] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:07:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:41.733976 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73f940-5d97-45de-825b-32c2cef2fd27 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:41.736845 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:41.843646 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73f940-5d97-45de-825b-32c2cef2fd27 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:41.880912 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e572d7-b3c9-4fd8-ac5d-1939834abc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:41.882560 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e572d7-b3c9-4fd8-ac5d-1939834abc00 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:41.883116 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e572d7-b3c9-4fd8-ac5d-1939834abc00 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:41.883463 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e572d7-b3c9-4fd8-ac5d-1939834abc00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:41.890936 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e572d7-b3c9-4fd8-ac5d-1939834abc00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:41.893165 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e572d7-b3c9-4fd8-ac5d-1939834abc00 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:41.908018 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-63e572d7-b3c9-4fd8-ac5d-1939834abc00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:07:41.908018 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:41.908018 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:41.908018 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:41.908018 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:41.908018 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:41.908018 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:41.908018 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:41.908018 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:41.908018 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:41.908018 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:41.908018 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:41.908018 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 06:07:41.908018 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 06:07:41.908018 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 06:07:41.908975 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 06:07:41.908975 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:41.908975 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:41.908975 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:41.908975 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:41.908975 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:41.908975 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:41.908975 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:41.908975 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:41.908975 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:41.908975 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:41.908975 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:41.908975 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:41.908975 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:41.908975 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:41.908975 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:41.908975 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:41.910176 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 06:07:41.910176 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 06:07:41.910176 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 06:07:41.910176 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 06:07:41.910176 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 06:07:41.910176 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 06:07:41.910176 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:07:41.910176 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:07:41.910176 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 542/1104] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:07:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:41.915822 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b6dd0-0ef9-4d72-b09a-d00440b7199a None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:42.028922 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b6dd0-0ef9-4d72-b09a-d00440b7199a None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:42.054662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:42.065486 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9956ac3-d50e-44a6-be70-cadade545098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:42.076625 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9956ac3-d50e-44a6-be70-cadade545098 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:42.076625 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9956ac3-d50e-44a6-be70-cadade545098 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:42.076625 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9956ac3-d50e-44a6-be70-cadade545098 None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:42.076625 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9956ac3-d50e-44a6-be70-cadade545098 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:42.076625 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9956ac3-d50e-44a6-be70-cadade545098 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:42.076625 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9956ac3-d50e-44a6-be70-cadade545098 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:42.082500 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 563/1105] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 18 06:07:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:42.090998 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e572d7-b3c9-4fd8-ac5d-1939834abc00 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:42.159712 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:42.200905 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e572d7-b3c9-4fd8-ac5d-1939834abc00 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:42.236823 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be821a01-758d-4872-8af9-a92ba384a396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:42.237481 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be821a01-758d-4872-8af9-a92ba384a396 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:42.237766 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be821a01-758d-4872-8af9-a92ba384a396 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:42.238045 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be821a01-758d-4872-8af9-a92ba384a396 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:42.238886 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be821a01-758d-4872-8af9-a92ba384a396 None admin] RBAC: Adding query filter params (role.id=34f769e98e1e449cb45aa4b31836393a, scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a, user.id=f92d948810ea4241810473506b654f7a) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:42.239203 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be821a01-758d-4872-8af9-a92ba384a396 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:42.240765 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be821a01-758d-4872-8af9-a92ba384a396 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:42.247931 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 543/1106] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:07:42 2022] GET /identity/v3/role_assignments?user.id=f92d948810ea4241810473506b654f7a&scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a&role.id=34f769e98e1e449cb45aa4b31836393a => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:42.443219 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449523f-e5aa-46ee-8e4d-c8d2c38b9c37 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:42.443494 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449523f-e5aa-46ee-8e4d-c8d2c38b9c37 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:42.444678 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449523f-e5aa-46ee-8e4d-c8d2c38b9c37 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:42.445797 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 564/1107] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:07:42.451684 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973f4e7-0c23-4c4d-bb5a-eded1caf8e15 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:42.451978 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973f4e7-0c23-4c4d-bb5a-eded1caf8e15 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:42.452268 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973f4e7-0c23-4c4d-bb5a-eded1caf8e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:42.471098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-0973f4e7-0c23-4c4d-bb5a-eded1caf8e15 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:42.472338 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-0973f4e7-0c23-4c4d-bb5a-eded1caf8e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:42.545631 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 544/1108] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:07:42.572078 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449523f-e5aa-46ee-8e4d-c8d2c38b9c37 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:42.698541 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449523f-e5aa-46ee-8e4d-c8d2c38b9c37 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:42.736900 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9378341-b515-4216-b5b5-7d437575723b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:42.737702 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9378341-b515-4216-b5b5-7d437575723b None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:42.737984 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9378341-b515-4216-b5b5-7d437575723b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:42.738260 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9378341-b515-4216-b5b5-7d437575723b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:42.739467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9378341-b515-4216-b5b5-7d437575723b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:42.741124 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9378341-b515-4216-b5b5-7d437575723b None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:42.742792 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 565/1109] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:07:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:42.749043 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0973f4e7-0c23-4c4d-bb5a-eded1caf8e15 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:42.817071 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:42.867146 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0973f4e7-0c23-4c4d-bb5a-eded1caf8e15 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:42.903921 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5314842-f284-4673-94cb-8cd26e33c30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:42.904652 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5314842-f284-4673-94cb-8cd26e33c30a None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:42.904935 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5314842-f284-4673-94cb-8cd26e33c30a None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:42.905210 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5314842-f284-4673-94cb-8cd26e33c30a None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:42.905975 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5314842-f284-4673-94cb-8cd26e33c30a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:42.907461 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5314842-f284-4673-94cb-8cd26e33c30a None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:43.051168 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 545/1110] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Jul 18 06:07:42 2022] POST /identity/v3/users => generated 286 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:07:45.129730 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1fca2-8950-4082-89bf-4924f16dea58 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:45.130360 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1fca2-8950-4082-89bf-4924f16dea58 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:45.130456 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1fca2-8950-4082-89bf-4924f16dea58 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:45.132394 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 566/1111] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:07:45.137797 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4984e9-bb6e-40e0-9067-8db88970386d None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:45.138237 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4984e9-bb6e-40e0-9067-8db88970386d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:45.138366 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4984e9-bb6e-40e0-9067-8db88970386d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:45.155750 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-de4984e9-bb6e-40e0-9067-8db88970386d None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:45.156995 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-de4984e9-bb6e-40e0-9067-8db88970386d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:45.226086 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 546/1112] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:07:45.250222 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d1fca2-8950-4082-89bf-4924f16dea58 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:45.306396 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:45.354291 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d1fca2-8950-4082-89bf-4924f16dea58 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:45.389014 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e6a05-cb3e-4b45-8cfd-96f1364ce169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:45.389779 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e6a05-cb3e-4b45-8cfd-96f1364ce169 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:45.390062 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e6a05-cb3e-4b45-8cfd-96f1364ce169 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:45.390340 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e6a05-cb3e-4b45-8cfd-96f1364ce169 None admin] PATH_INFO: `/v3/roles/04868303225144c389068779288f3a42` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:45.391818 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9e6a05-cb3e-4b45-8cfd-96f1364ce169 None admin] RBAC: Authorizing `identity:get_role(role_id=04868303225144c389068779288f3a42)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:45.393226 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9e6a05-cb3e-4b45-8cfd-96f1364ce169 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:45.394612 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 567/1113] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:07:45 2022] GET /identity/v3/roles/04868303225144c389068779288f3a42 => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:45.400569 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4984e9-bb6e-40e0-9067-8db88970386d None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:45.444786 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:45.503021 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4984e9-bb6e-40e0-9067-8db88970386d None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:45.539568 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b8b10-5284-4add-b8b8-161ee595574b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:45.540538 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b8b10-5284-4add-b8b8-161ee595574b None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:45.540827 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b8b10-5284-4add-b8b8-161ee595574b None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:45.541103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b8b10-5284-4add-b8b8-161ee595574b None admin] PATH_INFO: `/v3/users/d5dd3916c30c4927a4c474179ecbafe7` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:45.554689 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65b8b10-5284-4add-b8b8-161ee595574b None admin] RBAC: Authorizing `identity:get_user(user_id=d5dd3916c30c4927a4c474179ecbafe7)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:45.556134 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65b8b10-5284-4add-b8b8-161ee595574b None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:45.557995 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 547/1114] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:07:45 2022] GET /identity/v3/users/d5dd3916c30c4927a4c474179ecbafe7 => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:45.564112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e6a05-cb3e-4b45-8cfd-96f1364ce169 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:45.670643 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e6a05-cb3e-4b45-8cfd-96f1364ce169 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:45.693084 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:45.707669 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b2dfd4-9b33-4c9e-9a15-925d05a7a0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:45.708391 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2dfd4-9b33-4c9e-9a15-925d05a7a0ca None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:45.708711 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2dfd4-9b33-4c9e-9a15-925d05a7a0ca None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:45.709998 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2dfd4-9b33-4c9e-9a15-925d05a7a0ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:45.780449 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b2dfd4-9b33-4c9e-9a15-925d05a7a0ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:45.781898 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b2dfd4-9b33-4c9e-9a15-925d05a7a0ca None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:45.884121 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 568/1115] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:07:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 06:07:45.890301 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b8b10-5284-4add-b8b8-161ee595574b None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:45.992993 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:45.996059 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b8b10-5284-4add-b8b8-161ee595574b None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:46.030668 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9eda0d-7253-494f-9fc0-77aec4112656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:46.031532 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9eda0d-7253-494f-9fc0-77aec4112656 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:46.031822 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9eda0d-7253-494f-9fc0-77aec4112656 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:46.032099 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9eda0d-7253-494f-9fc0-77aec4112656 None admin] PATH_INFO: `/v3/projects/72e5dfa90eb34aabb7d344eff121a0e5` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:46.033314 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9eda0d-7253-494f-9fc0-77aec4112656 None admin] RBAC: Authorizing `identity:get_project(project_id=72e5dfa90eb34aabb7d344eff121a0e5)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:46.034756 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9eda0d-7253-494f-9fc0-77aec4112656 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:46.037056 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 548/1116] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:07:45 2022] GET /identity/v3/projects/72e5dfa90eb34aabb7d344eff121a0e5 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:46.043733 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b2dfd4-9b33-4c9e-9a15-925d05a7a0ca None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:46.066158 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:46.147205 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b2dfd4-9b33-4c9e-9a15-925d05a7a0ca None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:46.182736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a7815-ee7d-49bb-826d-fbc4b606023d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:46.183395 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a7815-ee7d-49bb-826d-fbc4b606023d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:46.183682 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a7815-ee7d-49bb-826d-fbc4b606023d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:46.183964 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a7815-ee7d-49bb-826d-fbc4b606023d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:46.184858 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a7815-ee7d-49bb-826d-fbc4b606023d None admin] RBAC: Adding query filter params (role.id=04868303225144c389068779288f3a42, scope.project.id=72e5dfa90eb34aabb7d344eff121a0e5, user.id=d5dd3916c30c4927a4c474179ecbafe7) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:46.185179 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a7815-ee7d-49bb-826d-fbc4b606023d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:46.186614 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a7815-ee7d-49bb-826d-fbc4b606023d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:46.192208 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 569/1117] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:07:46 2022] GET /identity/v3/role_assignments?user.id=d5dd3916c30c4927a4c474179ecbafe7&scope.project.id=72e5dfa90eb34aabb7d344eff121a0e5&role.id=04868303225144c389068779288f3a42 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:48.212780 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e4285-342f-4a55-9dbf-0cbe76732bec None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:48.213391 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e4285-342f-4a55-9dbf-0cbe76732bec None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:48.213391 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e4285-342f-4a55-9dbf-0cbe76732bec None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:48.214795 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 549/1118] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:07:48.219996 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300dfa49-f6c1-441b-b7ed-21a9fd2743f8 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:48.220447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300dfa49-f6c1-441b-b7ed-21a9fd2743f8 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:48.220589 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300dfa49-f6c1-441b-b7ed-21a9fd2743f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:48.237520 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-300dfa49-f6c1-441b-b7ed-21a9fd2743f8 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:48.238705 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-300dfa49-f6c1-441b-b7ed-21a9fd2743f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:48.307953 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 570/1119] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:07:48.330781 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957e4285-342f-4a55-9dbf-0cbe76732bec None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:48.436844 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957e4285-342f-4a55-9dbf-0cbe76732bec None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:48.470474 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc542e7-b809-4099-a574-df8a933feb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:48.471359 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc542e7-b809-4099-a574-df8a933feb67 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:48.471647 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc542e7-b809-4099-a574-df8a933feb67 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:48.471926 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc542e7-b809-4099-a574-df8a933feb67 None admin] PATH_INFO: `/v3/roles/04868303225144c389068779288f3a42` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:48.473397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc542e7-b809-4099-a574-df8a933feb67 None admin] RBAC: Authorizing `identity:get_role(role_id=04868303225144c389068779288f3a42)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:48.475251 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc542e7-b809-4099-a574-df8a933feb67 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:48.476917 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 550/1120] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:07:48 2022] GET /identity/v3/roles/04868303225144c389068779288f3a42 => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:48.482920 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300dfa49-f6c1-441b-b7ed-21a9fd2743f8 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:48.499115 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:48.582175 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300dfa49-f6c1-441b-b7ed-21a9fd2743f8 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:48.632394 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208895c-aa02-428a-a9ed-9501e3d71f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:48.633289 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208895c-aa02-428a-a9ed-9501e3d71f82 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:48.633573 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208895c-aa02-428a-a9ed-9501e3d71f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:48.633848 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208895c-aa02-428a-a9ed-9501e3d71f82 None admin] PATH_INFO: `/v3/users/d5dd3916c30c4927a4c474179ecbafe7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:48.635032 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0208895c-aa02-428a-a9ed-9501e3d71f82 None admin] RBAC: Authorizing `identity:get_user(user_id=d5dd3916c30c4927a4c474179ecbafe7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:48.636504 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0208895c-aa02-428a-a9ed-9501e3d71f82 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:48.638309 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 571/1121] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:07:48 2022] GET /identity/v3/users/d5dd3916c30c4927a4c474179ecbafe7 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:48.644382 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc542e7-b809-4099-a574-df8a933feb67 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:48.683433 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:48.749482 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc542e7-b809-4099-a574-df8a933feb67 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:48.783729 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c7bdc-283f-40ad-83d9-985469e749d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:48.784656 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c7bdc-283f-40ad-83d9-985469e749d6 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:48.785196 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c7bdc-283f-40ad-83d9-985469e749d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:48.785476 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c7bdc-283f-40ad-83d9-985469e749d6 None admin] PATH_INFO: `/v3/projects/72e5dfa90eb34aabb7d344eff121a0e5` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:48.786653 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0c7bdc-283f-40ad-83d9-985469e749d6 None admin] RBAC: Authorizing `identity:get_project(project_id=72e5dfa90eb34aabb7d344eff121a0e5)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:48.788871 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0c7bdc-283f-40ad-83d9-985469e749d6 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:48.790622 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 551/1122] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:07:48 2022] GET /identity/v3/projects/72e5dfa90eb34aabb7d344eff121a0e5 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:48.796767 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208895c-aa02-428a-a9ed-9501e3d71f82 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:48.882011 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:48.899952 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208895c-aa02-428a-a9ed-9501e3d71f82 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:48.934530 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c19006-c0f0-4d3a-9faa-93635aafb5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:48.935434 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c19006-c0f0-4d3a-9faa-93635aafb5ea None admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:48.935725 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c19006-c0f0-4d3a-9faa-93635aafb5ea None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:48.936006 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c19006-c0f0-4d3a-9faa-93635aafb5ea None admin] PATH_INFO: `/v3/projects/72e5dfa90eb34aabb7d344eff121a0e5/users/d5dd3916c30c4927a4c474179ecbafe7/roles/04868303225144c389068779288f3a42` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:48.938241 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c19006-c0f0-4d3a-9faa-93635aafb5ea None admin] RBAC: Authorizing `identity:create_grant(project_id=72e5dfa90eb34aabb7d344eff121a0e5, user_id=d5dd3916c30c4927a4c474179ecbafe7, role_id=04868303225144c389068779288f3a42)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:48.939682 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c19006-c0f0-4d3a-9faa-93635aafb5ea None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:49.033389 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 572/1123] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 18 06:07:48 2022] PUT /identity/v3/projects/72e5dfa90eb34aabb7d344eff121a0e5/users/d5dd3916c30c4927a4c474179ecbafe7/roles/04868303225144c389068779288f3a42 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:07:51.080168 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37c897-1734-4b9d-a297-578affbda2e7 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:51.080715 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37c897-1734-4b9d-a297-578affbda2e7 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:51.080715 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37c897-1734-4b9d-a297-578affbda2e7 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:51.082045 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 552/1124] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:07:51.088050 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56314bd-9eef-4b47-9787-3563a9101e67 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:51.088476 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56314bd-9eef-4b47-9787-3563a9101e67 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:51.088670 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56314bd-9eef-4b47-9787-3563a9101e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:51.105861 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-d56314bd-9eef-4b47-9787-3563a9101e67 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:51.107078 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d56314bd-9eef-4b47-9787-3563a9101e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:51.176753 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 573/1125] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:07:51.201447 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37c897-1734-4b9d-a297-578affbda2e7 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:51.307487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37c897-1734-4b9d-a297-578affbda2e7 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:51.309990 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:51.343313 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974addd2-a6cd-4682-abe9-ce4427c17b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:51.344111 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974addd2-a6cd-4682-abe9-ce4427c17b85 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:51.344397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974addd2-a6cd-4682-abe9-ce4427c17b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:51.344706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974addd2-a6cd-4682-abe9-ce4427c17b85 None admin] PATH_INFO: `/v3/roles/04868303225144c389068779288f3a42` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:51.346286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974addd2-a6cd-4682-abe9-ce4427c17b85 None admin] RBAC: Authorizing `identity:get_role(role_id=04868303225144c389068779288f3a42)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:51.347650 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974addd2-a6cd-4682-abe9-ce4427c17b85 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:51.349114 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 553/1126] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:07:51 2022] GET /identity/v3/roles/04868303225144c389068779288f3a42 => generated 223 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:51.355543 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56314bd-9eef-4b47-9787-3563a9101e67 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:51.431992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:51.462145 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56314bd-9eef-4b47-9787-3563a9101e67 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:51.495346 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10d0563-d342-46f1-81fd-151c1270289f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:51.496365 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d0563-d342-46f1-81fd-151c1270289f None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:51.497099 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d0563-d342-46f1-81fd-151c1270289f None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:51.497379 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d0563-d342-46f1-81fd-151c1270289f None admin] PATH_INFO: `/v3/users/d5dd3916c30c4927a4c474179ecbafe7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:51.498577 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10d0563-d342-46f1-81fd-151c1270289f None admin] RBAC: Authorizing `identity:get_user(user_id=d5dd3916c30c4927a4c474179ecbafe7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:51.500029 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10d0563-d342-46f1-81fd-151c1270289f None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:51.501720 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 574/1127] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:07:51 2022] GET /identity/v3/users/d5dd3916c30c4927a4c474179ecbafe7 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:51.508605 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974addd2-a6cd-4682-abe9-ce4427c17b85 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:51.623405 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974addd2-a6cd-4682-abe9-ce4427c17b85 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:51.657337 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e6065-cc35-48bb-8c30-5b6bf812384d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:51.657988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e6065-cc35-48bb-8c30-5b6bf812384d None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:51.658276 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e6065-cc35-48bb-8c30-5b6bf812384d None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:51.658554 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e6065-cc35-48bb-8c30-5b6bf812384d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:51.701509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:51.727436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7e6065-cc35-48bb-8c30-5b6bf812384d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:51.728920 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7e6065-cc35-48bb-8c30-5b6bf812384d None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:51.830337 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 554/1128] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:07:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 06:07:51.836551 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10d0563-d342-46f1-81fd-151c1270289f None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:51.946569 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10d0563-d342-46f1-81fd-151c1270289f None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:51.982889 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f297f7f-d93b-49da-ada0-4f0f0a4114ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:51.983764 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f297f7f-d93b-49da-ada0-4f0f0a4114ca None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:51.984053 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f297f7f-d93b-49da-ada0-4f0f0a4114ca None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:51.984334 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f297f7f-d93b-49da-ada0-4f0f0a4114ca None admin] PATH_INFO: `/v3/projects/72e5dfa90eb34aabb7d344eff121a0e5` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:51.985568 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f297f7f-d93b-49da-ada0-4f0f0a4114ca None admin] RBAC: Authorizing `identity:get_project(project_id=72e5dfa90eb34aabb7d344eff121a0e5)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:51.987042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f297f7f-d93b-49da-ada0-4f0f0a4114ca None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:51.988802 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 575/1129] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:07:51 2022] GET /identity/v3/projects/72e5dfa90eb34aabb7d344eff121a0e5 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:51.995686 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e6065-cc35-48bb-8c30-5b6bf812384d None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:52.069134 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:52.097884 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e6065-cc35-48bb-8c30-5b6bf812384d None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:52.135115 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d145f-4975-444a-97f7-fb54e0ea0bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:52.135769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d145f-4975-444a-97f7-fb54e0ea0bbc None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:52.136053 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d145f-4975-444a-97f7-fb54e0ea0bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:52.136333 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d145f-4975-444a-97f7-fb54e0ea0bbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:52.137200 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d145f-4975-444a-97f7-fb54e0ea0bbc None admin] RBAC: Adding query filter params (role.id=04868303225144c389068779288f3a42, scope.project.id=72e5dfa90eb34aabb7d344eff121a0e5, user.id=d5dd3916c30c4927a4c474179ecbafe7) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:52.137513 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d145f-4975-444a-97f7-fb54e0ea0bbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:52.138934 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d145f-4975-444a-97f7-fb54e0ea0bbc None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:52.144846 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 555/1130] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:07:51 2022] GET /identity/v3/role_assignments?user.id=d5dd3916c30c4927a4c474179ecbafe7&scope.project.id=72e5dfa90eb34aabb7d344eff121a0e5&role.id=04868303225144c389068779288f3a42 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:54.179658 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4545e-695d-4faf-a165-32d83dfe3343 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:54.180221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4545e-695d-4faf-a165-32d83dfe3343 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:54.180221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4545e-695d-4faf-a165-32d83dfe3343 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:54.181623 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 576/1131] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:07:54.186876 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe23db-20b4-4e95-a923-0dbeccf10729 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:54.187301 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe23db-20b4-4e95-a923-0dbeccf10729 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:54.187429 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe23db-20b4-4e95-a923-0dbeccf10729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:54.204802 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-10fe23db-20b4-4e95-a923-0dbeccf10729 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:54.205996 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-10fe23db-20b4-4e95-a923-0dbeccf10729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:54.274517 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 556/1132] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:07:54.298569 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce4545e-695d-4faf-a165-32d83dfe3343 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:54.310690 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:54.404643 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce4545e-695d-4faf-a165-32d83dfe3343 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:54.439119 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24646fad-7061-446e-a86a-57b0c9cceab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:54.439820 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24646fad-7061-446e-a86a-57b0c9cceab1 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:54.440102 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24646fad-7061-446e-a86a-57b0c9cceab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:54.440378 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24646fad-7061-446e-a86a-57b0c9cceab1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:54.441566 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24646fad-7061-446e-a86a-57b0c9cceab1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:54.443005 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24646fad-7061-446e-a86a-57b0c9cceab1 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:54.445097 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 577/1133] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:07:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:54.451444 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fe23db-20b4-4e95-a923-0dbeccf10729 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:54.520988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:54.556260 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fe23db-20b4-4e95-a923-0dbeccf10729 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:54.590259 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fda459-e12f-492b-b3d2-75f9206cc950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:54.590918 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fda459-e12f-492b-b3d2-75f9206cc950 None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:54.591203 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fda459-e12f-492b-b3d2-75f9206cc950 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:54.591483 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fda459-e12f-492b-b3d2-75f9206cc950 None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:54.592202 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fda459-e12f-492b-b3d2-75f9206cc950 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:54.593664 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fda459-e12f-492b-b3d2-75f9206cc950 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:54.684785 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 557/1134] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Jul 18 06:07:54 2022] POST /identity/v3/projects => generated 311 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:07:56.783458 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c9ad4-7237-4c7f-ba14-72d862e339ad None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:56.784018 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c9ad4-7237-4c7f-ba14-72d862e339ad None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:56.784018 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c9ad4-7237-4c7f-ba14-72d862e339ad None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:56.785731 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 578/1135] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:07:56.791775 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1fb4dc-c00a-4fcd-8139-c7ac19611131 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:56.792195 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1fb4dc-c00a-4fcd-8139-c7ac19611131 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:56.792332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1fb4dc-c00a-4fcd-8139-c7ac19611131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:56.809865 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-7f1fb4dc-c00a-4fcd-8139-c7ac19611131 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:56.811047 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7f1fb4dc-c00a-4fcd-8139-c7ac19611131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:56.880074 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 558/1136] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 06:07:56.905592 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c9ad4-7237-4c7f-ba14-72d862e339ad None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:56.984498 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:57.008805 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c9ad4-7237-4c7f-ba14-72d862e339ad None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:57.042485 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af084c0-6774-481b-b1f2-355b84b36071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:57.043284 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af084c0-6774-481b-b1f2-355b84b36071 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:57.043576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af084c0-6774-481b-b1f2-355b84b36071 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:57.043859 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af084c0-6774-481b-b1f2-355b84b36071 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:57.045067 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af084c0-6774-481b-b1f2-355b84b36071 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:57.046512 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af084c0-6774-481b-b1f2-355b84b36071 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:57.048136 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 579/1137] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:07:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:57.054940 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1fb4dc-c00a-4fcd-8139-c7ac19611131 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:57.103302 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:57.153381 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1fb4dc-c00a-4fcd-8139-c7ac19611131 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:57.188747 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474cfc1-7563-4b03-84aa-8c836cddbb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:57.189422 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474cfc1-7563-4b03-84aa-8c836cddbb41 None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:57.189708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474cfc1-7563-4b03-84aa-8c836cddbb41 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:57.189985 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474cfc1-7563-4b03-84aa-8c836cddbb41 None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:57.190699 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f474cfc1-7563-4b03-84aa-8c836cddbb41 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:57.192162 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f474cfc1-7563-4b03-84aa-8c836cddbb41 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:57.283318 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 559/1138] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Jul 18 06:07:57 2022] POST /identity/v3/users => generated 286 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:07:59.498392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8fc289-f2f3-4c79-a81c-ce96668457f3 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:59.498997 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8fc289-f2f3-4c79-a81c-ce96668457f3 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:59.498997 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8fc289-f2f3-4c79-a81c-ce96668457f3 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:59.500295 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 580/1139] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:07:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:07:59.506281 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548007a-2507-4fde-af63-229c94852773 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:59.506752 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548007a-2507-4fde-af63-229c94852773 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:59.506908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548007a-2507-4fde-af63-229c94852773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:59.524674 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-2548007a-2507-4fde-af63-229c94852773 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:07:59.525921 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-2548007a-2507-4fde-af63-229c94852773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:07:59.594159 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 560/1140] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:07:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:07:59.618568 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8fc289-f2f3-4c79-a81c-ce96668457f3 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:59.723602 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8fc289-f2f3-4c79-a81c-ce96668457f3 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:59.758982 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a8e3c-8c77-40c2-a38a-012a3711d596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:59.759698 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a8e3c-8c77-40c2-a38a-012a3711d596 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:59.759988 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a8e3c-8c77-40c2-a38a-012a3711d596 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:59.760273 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a8e3c-8c77-40c2-a38a-012a3711d596 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:59.771670 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9a8e3c-8c77-40c2-a38a-012a3711d596 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:07:59.779239 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-eb9a8e3c-8c77-40c2-a38a-012a3711d596 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:07:59.779239 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:07:59.779239 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:07:59.779239 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:07:59.779239 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:07:59.779239 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:07:59.779239 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:07:59.779239 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:07:59.779239 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:07:59.779239 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:07:59.779239 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:07:59.779239 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:07:59.779239 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:07:59.779239 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:07:59.779239 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:07:59.779239 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:07:59.780389 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:07:59.780389 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:07:59.780389 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:07:59.780389 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:07:59.780389 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:07:59.780389 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:07:59.780389 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:07:59.780389 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:07:59.780389 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:07:59.780389 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:07:59.780389 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:07:59.780389 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:07:59.780389 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:07:59.780389 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:07:59.780389 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:07:59.780389 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:07:59.781335 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:07:59.781335 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:07:59.781335 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:07:59.781335 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:07:59.781335 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:07:59.781335 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:07:59.781335 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:07:59.781335 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:07:59.781335 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:07:59.781335 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:07:59.781335 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:07:59.781335 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:07:59.781335 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 581/1141] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 18 06:07:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:07:59.787552 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2548007a-2507-4fde-af63-229c94852773 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:59.820712 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:07:59.890066 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2548007a-2507-4fde-af63-229c94852773 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:07:59.927001 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54de29d-f906-4dd8-9a6c-3cdbaa63c5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:07:59.928054 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54de29d-f906-4dd8-9a6c-3cdbaa63c5d7 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:07:59.928353 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54de29d-f906-4dd8-9a6c-3cdbaa63c5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:07:59.928773 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54de29d-f906-4dd8-9a6c-3cdbaa63c5d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:07:59.929703 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54de29d-f906-4dd8-9a6c-3cdbaa63c5d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:07:59.930017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54de29d-f906-4dd8-9a6c-3cdbaa63c5d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:07:59.931397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54de29d-f906-4dd8-9a6c-3cdbaa63c5d7 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:07:59.940249 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 561/1142] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:07:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:07:59.947303 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a8e3c-8c77-40c2-a38a-012a3711d596 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:07:59.953641 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:08:00.073491 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a8e3c-8c77-40c2-a38a-012a3711d596 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:00.110810 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bd580d-ab36-4574-80d0-cff9cd12a484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:00.111682 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bd580d-ab36-4574-80d0-cff9cd12a484 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:00.111963 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bd580d-ab36-4574-80d0-cff9cd12a484 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:00.112236 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bd580d-ab36-4574-80d0-cff9cd12a484 None admin] PATH_INFO: `/v3/users/d2899761f08340e3bf6209bd2a4b80d3` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:00.126063 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bd580d-ab36-4574-80d0-cff9cd12a484 None admin] RBAC: Authorizing `identity:get_user(user_id=d2899761f08340e3bf6209bd2a4b80d3)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:08:00.127516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bd580d-ab36-4574-80d0-cff9cd12a484 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:08:00.130271 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 582/1143] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:07:59 2022] GET /identity/v3/users/d2899761f08340e3bf6209bd2a4b80d3 => generated 286 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:08:00.137548 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54de29d-f906-4dd8-9a6c-3cdbaa63c5d7 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:00.231094 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:08:00.236701 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54de29d-f906-4dd8-9a6c-3cdbaa63c5d7 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:00.269458 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ff13d-4130-4202-8aff-5d92e1af0c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:00.270091 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ff13d-4130-4202-8aff-5d92e1af0c28 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:00.270376 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ff13d-4130-4202-8aff-5d92e1af0c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:00.270652 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ff13d-4130-4202-8aff-5d92e1af0c28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:00.339441 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021ff13d-4130-4202-8aff-5d92e1af0c28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:08:00.340812 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021ff13d-4130-4202-8aff-5d92e1af0c28 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:08:00.439345 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:08:00.446885 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 562/1144] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:08:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 06:08:00.453400 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bd580d-ab36-4574-80d0-cff9cd12a484 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:00.519488 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:08:00.560751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bd580d-ab36-4574-80d0-cff9cd12a484 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:00.596598 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22692d3-c8ad-485a-b195-9663853d79af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:00.597375 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22692d3-c8ad-485a-b195-9663853d79af None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:00.597667 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22692d3-c8ad-485a-b195-9663853d79af None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:00.597983 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22692d3-c8ad-485a-b195-9663853d79af None admin] PATH_INFO: `/v3/projects/a6ec91ab34734d6db5179b7e6dc09637` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:00.599314 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22692d3-c8ad-485a-b195-9663853d79af None admin] RBAC: Authorizing `identity:get_project(project_id=a6ec91ab34734d6db5179b7e6dc09637)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:08:00.601285 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22692d3-c8ad-485a-b195-9663853d79af None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:08:00.603048 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 583/1145] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:08:00 2022] GET /identity/v3/projects/a6ec91ab34734d6db5179b7e6dc09637 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:08:00.609410 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ff13d-4130-4202-8aff-5d92e1af0c28 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:00.714144 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ff13d-4130-4202-8aff-5d92e1af0c28 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:00.750766 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8318a8-f3dd-4e59-afae-e280381dd4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:00.751844 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8318a8-f3dd-4e59-afae-e280381dd4ee None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:00.752345 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8318a8-f3dd-4e59-afae-e280381dd4ee None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:00.752871 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8318a8-f3dd-4e59-afae-e280381dd4ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:00.754441 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8318a8-f3dd-4e59-afae-e280381dd4ee None admin] RBAC: Adding query filter params (role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc, scope.project.id=a6ec91ab34734d6db5179b7e6dc09637, user.id=d2899761f08340e3bf6209bd2a4b80d3) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:08:00.754995 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8318a8-f3dd-4e59-afae-e280381dd4ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:08:00.757009 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8318a8-f3dd-4e59-afae-e280381dd4ee None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:08:00.762842 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 563/1146] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:08:00 2022] GET /identity/v3/role_assignments?user.id=d2899761f08340e3bf6209bd2a4b80d3&scope.project.id=a6ec91ab34734d6db5179b7e6dc09637&role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:08:02.788700 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc9e7c-8b89-43a9-8383-d9e27c88b222 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:02.789298 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc9e7c-8b89-43a9-8383-d9e27c88b222 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:02.789298 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc9e7c-8b89-43a9-8383-d9e27c88b222 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:02.790648 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 584/1147] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:08:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:08:02.795929 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedbe603-3f76-4b8b-a925-c2d6574f7955 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:02.796711 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedbe603-3f76-4b8b-a925-c2d6574f7955 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:02.796711 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedbe603-3f76-4b8b-a925-c2d6574f7955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:02.814108 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-aedbe603-3f76-4b8b-a925-c2d6574f7955 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:08:02.815363 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-aedbe603-3f76-4b8b-a925-c2d6574f7955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:08:02.873358 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:08:02.885317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 564/1148] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:08:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:08:02.908249 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc9e7c-8b89-43a9-8383-d9e27c88b222 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:03.011818 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc9e7c-8b89-43a9-8383-d9e27c88b222 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:03.041382 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:08:03.047130 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326da66f-ab97-4415-a7fa-77a4600be660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:03.047948 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326da66f-ab97-4415-a7fa-77a4600be660 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:03.050285 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326da66f-ab97-4415-a7fa-77a4600be660 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:03.050285 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326da66f-ab97-4415-a7fa-77a4600be660 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:03.059707 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326da66f-ab97-4415-a7fa-77a4600be660 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:08:03.067373 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-326da66f-ab97-4415-a7fa-77a4600be660 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:08:03.067373 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:08:03.067373 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:08:03.067373 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:08:03.067373 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:08:03.067373 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:08:03.067373 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:08:03.067373 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:08:03.067373 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:08:03.067373 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:08:03.067373 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:08:03.067373 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:08:03.067373 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:08:03.067373 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:08:03.067373 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:08:03.067373 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:08:03.069654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:08:03.069654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:08:03.069654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:08:03.069654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:08:03.069654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:08:03.069654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:08:03.069654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:08:03.069654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:08:03.069654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:08:03.069654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:08:03.069654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:08:03.069654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:08:03.069654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:08:03.069654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:08:03.069654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:08:03.069654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:08:03.070841 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:08:03.070841 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:08:03.070841 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:08:03.070841 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:08:03.070841 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:08:03.070841 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:08:03.070841 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:08:03.070841 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:08:03.070841 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:08:03.070841 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:08:03.070841 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:08:03.070841 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:08:03.070841 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 585/1149] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 18 06:08:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:08:03.075532 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedbe603-3f76-4b8b-a925-c2d6574f7955 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:03.184754 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedbe603-3f76-4b8b-a925-c2d6574f7955 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:03.220529 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47a1a9-6e36-4b8c-8917-fadbfa75064d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:03.221200 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47a1a9-6e36-4b8c-8917-fadbfa75064d None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:03.221490 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47a1a9-6e36-4b8c-8917-fadbfa75064d None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:03.221773 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47a1a9-6e36-4b8c-8917-fadbfa75064d None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:03.222525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc47a1a9-6e36-4b8c-8917-fadbfa75064d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:08:03.222836 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc47a1a9-6e36-4b8c-8917-fadbfa75064d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:08:03.224928 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc47a1a9-6e36-4b8c-8917-fadbfa75064d None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:08:03.233470 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 565/1150] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:08:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:08:03.240451 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326da66f-ab97-4415-a7fa-77a4600be660 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:03.347938 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326da66f-ab97-4415-a7fa-77a4600be660 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:03.383829 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e3e87-8f0e-4010-81ef-8f59b020956c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:03.384740 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e3e87-8f0e-4010-81ef-8f59b020956c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:03.385027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e3e87-8f0e-4010-81ef-8f59b020956c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:03.385304 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e3e87-8f0e-4010-81ef-8f59b020956c None admin] PATH_INFO: `/v3/users/d2899761f08340e3bf6209bd2a4b80d3` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:03.386494 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e3e87-8f0e-4010-81ef-8f59b020956c None admin] RBAC: Authorizing `identity:get_user(user_id=d2899761f08340e3bf6209bd2a4b80d3)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:08:03.387953 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e3e87-8f0e-4010-81ef-8f59b020956c None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:08:03.389626 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 586/1151] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:08:03 2022] GET /identity/v3/users/d2899761f08340e3bf6209bd2a4b80d3 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:08:03.395587 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47a1a9-6e36-4b8c-8917-fadbfa75064d None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:03.450415 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:08:03.499436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47a1a9-6e36-4b8c-8917-fadbfa75064d None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:03.534641 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecc4035-9930-40cd-942d-3f9462bfa88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:03.535668 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc4035-9930-40cd-942d-3f9462bfa88b None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:03.535961 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc4035-9930-40cd-942d-3f9462bfa88b None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:03.536244 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc4035-9930-40cd-942d-3f9462bfa88b None admin] PATH_INFO: `/v3/projects/a6ec91ab34734d6db5179b7e6dc09637` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:03.538098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecc4035-9930-40cd-942d-3f9462bfa88b None admin] RBAC: Authorizing `identity:get_project(project_id=a6ec91ab34734d6db5179b7e6dc09637)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:08:03.539629 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecc4035-9930-40cd-942d-3f9462bfa88b None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:08:03.541546 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 566/1152] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:08:03 2022] GET /identity/v3/projects/a6ec91ab34734d6db5179b7e6dc09637 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:08:03.547738 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e3e87-8f0e-4010-81ef-8f59b020956c None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:03.608439 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:08:03.650991 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e3e87-8f0e-4010-81ef-8f59b020956c None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:03.688303 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b758a-fa63-4389-9c7d-31a1c4197855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:03.689241 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b758a-fa63-4389-9c7d-31a1c4197855 None admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:03.689527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b758a-fa63-4389-9c7d-31a1c4197855 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:03.689801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b758a-fa63-4389-9c7d-31a1c4197855 None admin] PATH_INFO: `/v3/projects/a6ec91ab34734d6db5179b7e6dc09637/users/d2899761f08340e3bf6209bd2a4b80d3/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:03.691744 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1b758a-fa63-4389-9c7d-31a1c4197855 None admin] RBAC: Authorizing `identity:create_grant(project_id=a6ec91ab34734d6db5179b7e6dc09637, user_id=d2899761f08340e3bf6209bd2a4b80d3, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:08:03.693388 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1b758a-fa63-4389-9c7d-31a1c4197855 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:08:03.788282 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 587/1153] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 18 06:08:03 2022] PUT /identity/v3/projects/a6ec91ab34734d6db5179b7e6dc09637/users/d2899761f08340e3bf6209bd2a4b80d3/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:08:05.815430 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a224f-77e0-4e1a-9076-cd6b0625344c None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:05.815994 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a224f-77e0-4e1a-9076-cd6b0625344c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:05.815994 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a224f-77e0-4e1a-9076-cd6b0625344c None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:05.817491 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 567/1154] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:08:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:08:05.822751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18aac19-9ba0-49c2-abe3-a56838de3686 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:05.823217 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18aac19-9ba0-49c2-abe3-a56838de3686 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:05.823463 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18aac19-9ba0-49c2-abe3-a56838de3686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:05.840810 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-f18aac19-9ba0-49c2-abe3-a56838de3686 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:08:05.841994 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-f18aac19-9ba0-49c2-abe3-a56838de3686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:08:05.908599 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 588/1155] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:08:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:08:05.933941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a224f-77e0-4e1a-9076-cd6b0625344c None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:06.034730 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a224f-77e0-4e1a-9076-cd6b0625344c None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:06.055858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:08:06.070198 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8bddad-1b1f-4d78-8690-5fd4fb560b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:06.073150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8bddad-1b1f-4d78-8690-5fd4fb560b66 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:06.073150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8bddad-1b1f-4d78-8690-5fd4fb560b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:06.073150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8bddad-1b1f-4d78-8690-5fd4fb560b66 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:06.081891 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8bddad-1b1f-4d78-8690-5fd4fb560b66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:08:06.089998 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-7f8bddad-1b1f-4d78-8690-5fd4fb560b66 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:08:06.089998 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:08:06.089998 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:08:06.089998 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:08:06.089998 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:08:06.089998 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:08:06.089998 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:08:06.089998 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:08:06.089998 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:08:06.089998 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:08:06.089998 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:08:06.089998 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:08:06.089998 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:08:06.089998 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:08:06.089998 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:08:06.089998 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:08:06.091166 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:08:06.091166 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:08:06.091166 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:08:06.091166 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:08:06.091166 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:08:06.091166 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:08:06.091166 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:08:06.091166 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:08:06.091166 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:08:06.091166 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:08:06.091166 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:08:06.091166 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:08:06.091166 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:08:06.091166 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:08:06.091166 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:08:06.091166 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:08:06.092222 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:08:06.092222 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:08:06.092222 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:08:06.092222 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:08:06.092222 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:08:06.092222 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:08:06.092222 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:08:06.092222 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:08:06.092222 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:08:06.092222 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:08:06.092222 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:08:06.092222 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:08:06.092222 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 568/1156] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 18 06:08:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:08:06.097688 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18aac19-9ba0-49c2-abe3-a56838de3686 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:06.147183 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:08:06.198493 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18aac19-9ba0-49c2-abe3-a56838de3686 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:06.232738 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2cca82-6144-4d2a-b6cc-aa17d5374bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:06.233412 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2cca82-6144-4d2a-b6cc-aa17d5374bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:06.233699 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2cca82-6144-4d2a-b6cc-aa17d5374bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:06.233980 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2cca82-6144-4d2a-b6cc-aa17d5374bb2 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:06.234734 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2cca82-6144-4d2a-b6cc-aa17d5374bb2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:08:06.235043 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2cca82-6144-4d2a-b6cc-aa17d5374bb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:08:06.236557 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2cca82-6144-4d2a-b6cc-aa17d5374bb2 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:08:06.244764 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 589/1157] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:08:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:08:06.251065 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8bddad-1b1f-4d78-8690-5fd4fb560b66 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:06.356094 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8bddad-1b1f-4d78-8690-5fd4fb560b66 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:06.390149 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5255cdc4-b035-478d-8acc-073f14cc9980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:06.391076 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255cdc4-b035-478d-8acc-073f14cc9980 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:06.391361 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255cdc4-b035-478d-8acc-073f14cc9980 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:06.391643 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255cdc4-b035-478d-8acc-073f14cc9980 None admin] PATH_INFO: `/v3/users/d2899761f08340e3bf6209bd2a4b80d3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:06.392871 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5255cdc4-b035-478d-8acc-073f14cc9980 None admin] RBAC: Authorizing `identity:get_user(user_id=d2899761f08340e3bf6209bd2a4b80d3)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:08:06.394299 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5255cdc4-b035-478d-8acc-073f14cc9980 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:08:06.396137 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 569/1158] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:08:06 2022] GET /identity/v3/users/d2899761f08340e3bf6209bd2a4b80d3 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:08:06.402120 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2cca82-6144-4d2a-b6cc-aa17d5374bb2 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:06.504980 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2cca82-6144-4d2a-b6cc-aa17d5374bb2 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:06.515390 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:08:06.540446 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8271613b-a7cd-4f37-b2c0-a64d445e6a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:06.541107 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271613b-a7cd-4f37-b2c0-a64d445e6a08 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:06.541395 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271613b-a7cd-4f37-b2c0-a64d445e6a08 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:06.541668 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271613b-a7cd-4f37-b2c0-a64d445e6a08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:06.609061 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8271613b-a7cd-4f37-b2c0-a64d445e6a08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:08:06.610413 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8271613b-a7cd-4f37-b2c0-a64d445e6a08 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:08:06.711078 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 590/1159] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:08:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 06:08:06.717223 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5255cdc4-b035-478d-8acc-073f14cc9980 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:06.766964 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:08:06.819111 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5255cdc4-b035-478d-8acc-073f14cc9980 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:06.853879 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02be49a9-d3e9-4052-a36f-d3c021880ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:06.854741 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02be49a9-d3e9-4052-a36f-d3c021880ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:06.855025 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02be49a9-d3e9-4052-a36f-d3c021880ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:06.855305 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02be49a9-d3e9-4052-a36f-d3c021880ed1 None admin] PATH_INFO: `/v3/projects/a6ec91ab34734d6db5179b7e6dc09637` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:06.856536 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02be49a9-d3e9-4052-a36f-d3c021880ed1 None admin] RBAC: Authorizing `identity:get_project(project_id=a6ec91ab34734d6db5179b7e6dc09637)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:08:06.857972 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02be49a9-d3e9-4052-a36f-d3c021880ed1 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:08:06.859803 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 570/1160] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:08:06 2022] GET /identity/v3/projects/a6ec91ab34734d6db5179b7e6dc09637 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:08:06.866491 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8271613b-a7cd-4f37-b2c0-a64d445e6a08 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:06.878277 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:08:06.967096 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8271613b-a7cd-4f37-b2c0-a64d445e6a08 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:07.001075 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e753e69d-98e2-4437-a9ae-dee5c8005204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:07.001733 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753e69d-98e2-4437-a9ae-dee5c8005204 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:07.002015 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753e69d-98e2-4437-a9ae-dee5c8005204 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:07.002291 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753e69d-98e2-4437-a9ae-dee5c8005204 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:07.003122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e753e69d-98e2-4437-a9ae-dee5c8005204 None admin] RBAC: Adding query filter params (role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc, scope.project.id=a6ec91ab34734d6db5179b7e6dc09637, user.id=d2899761f08340e3bf6209bd2a4b80d3) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:08:07.003433 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e753e69d-98e2-4437-a9ae-dee5c8005204 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:08:07.004896 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e753e69d-98e2-4437-a9ae-dee5c8005204 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:08:07.011737 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 591/1161] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:08:06 2022] GET /identity/v3/role_assignments?user.id=d2899761f08340e3bf6209bd2a4b80d3&scope.project.id=a6ec91ab34734d6db5179b7e6dc09637&role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:08:09.076690 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65467e62-0621-4ffa-ad28-762f13cd287f None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:09.077213 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65467e62-0621-4ffa-ad28-762f13cd287f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:09.077213 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65467e62-0621-4ffa-ad28-762f13cd287f None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:09.078535 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 571/1162] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:08:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:08:09.084370 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70bda15-97f0-4510-a729-a7792c2c0d90 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:09.084836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70bda15-97f0-4510-a729-a7792c2c0d90 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:09.084969 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70bda15-97f0-4510-a729-a7792c2c0d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:09.102139 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-f70bda15-97f0-4510-a729-a7792c2c0d90 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:08:09.103375 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-f70bda15-97f0-4510-a729-a7792c2c0d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:08:09.170841 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 592/1163] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:08:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:08:09.194259 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65467e62-0621-4ffa-ad28-762f13cd287f None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:09.296344 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65467e62-0621-4ffa-ad28-762f13cd287f None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:09.313286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:08:09.332182 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07abda11-a61b-43db-af31-9f81081d536a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:09.332182 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07abda11-a61b-43db-af31-9f81081d536a None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:09.332182 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07abda11-a61b-43db-af31-9f81081d536a None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:09.332182 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07abda11-a61b-43db-af31-9f81081d536a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:09.398113 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07abda11-a61b-43db-af31-9f81081d536a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:08:09.399546 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07abda11-a61b-43db-af31-9f81081d536a None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:08:09.498606 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 572/1164] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:08:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 06:08:09.504934 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70bda15-97f0-4510-a729-a7792c2c0d90 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:09.508377 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:08:09.607494 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70bda15-97f0-4510-a729-a7792c2c0d90 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:09.642238 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3dcd99-e894-4cce-b7c1-598780395e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:09.643040 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3dcd99-e894-4cce-b7c1-598780395e7d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:09.643330 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3dcd99-e894-4cce-b7c1-598780395e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:09.643613 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3dcd99-e894-4cce-b7c1-598780395e7d None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:09.651316 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3dcd99-e894-4cce-b7c1-598780395e7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:08:09.652833 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3dcd99-e894-4cce-b7c1-598780395e7d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:08:09.668082 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-0b3dcd99-e894-4cce-b7c1-598780395e7d None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 18 06:08:09.668082 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:08:09.668082 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 18 06:08:09.668082 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 18 06:08:09.668082 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 06:08:09.668082 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 06:08:09.668082 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 06:08:09.668082 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 06:08:09.668082 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 18 06:08:09.668082 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:08:09.668082 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 06:08:09.668082 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:08:09.668082 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:08:09.668082 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:08:09.668082 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:08:09.668082 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:08:09.668082 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:08:09.669370 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:08:09.669370 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:08:09.669370 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:08:09.669370 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:08:09.669370 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:08:09.669370 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:08:09.669370 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:08:09.669370 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:08:09.669370 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:08:09.669370 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:08:09.669370 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:08:09.669370 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:08:09.669370 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:08:09.669370 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:08:09.669370 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:08:09.669370 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:08:09.670506 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:08:09.670506 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:08:09.670506 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:08:09.670506 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:08:09.670506 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:08:09.670506 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:08:09.670506 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:08:09.670506 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:08:09.670506 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:08:09.670506 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:08:09.670506 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 18 06:08:09.670506 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:08:09.670506 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 18 06:08:09.670506 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:08:09.670506 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 593/1165] 10.222.0.40 () {56 vars in 1117 bytes} [Mon Jul 18 06:08:09 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:08:09.675273 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07abda11-a61b-43db-af31-9f81081d536a None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:09.692867 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:08:09.778111 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07abda11-a61b-43db-af31-9f81081d536a None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:09.817274 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5098c-4efe-4c0f-9893-e6b40630fc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:09.818266 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5098c-4efe-4c0f-9893-e6b40630fc7e None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:09.818559 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5098c-4efe-4c0f-9893-e6b40630fc7e None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:09.818960 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5098c-4efe-4c0f-9893-e6b40630fc7e None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:09.819749 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5098c-4efe-4c0f-9893-e6b40630fc7e None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:08:09.820063 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5098c-4efe-4c0f-9893-e6b40630fc7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:08:09.821492 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5098c-4efe-4c0f-9893-e6b40630fc7e None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:08:09.829766 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 573/1166] 10.222.0.40 () {56 vars in 1115 bytes} [Mon Jul 18 06:08:09 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:08:11.861722 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d7c3b5-2b53-4b58-9eb3-11b70b67bc44 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:11.862279 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d7c3b5-2b53-4b58-9eb3-11b70b67bc44 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:11.862279 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d7c3b5-2b53-4b58-9eb3-11b70b67bc44 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:11.863643 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 594/1167] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:08:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:08:11.869856 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b16fdc-607d-4265-bd36-f66a94650804 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:11.870299 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b16fdc-607d-4265-bd36-f66a94650804 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:11.870403 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b16fdc-607d-4265-bd36-f66a94650804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:11.888840 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-43b16fdc-607d-4265-bd36-f66a94650804 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:08:11.890028 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-43b16fdc-607d-4265-bd36-f66a94650804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:08:11.957617 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 574/1168] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:08:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:08:11.980804 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d7c3b5-2b53-4b58-9eb3-11b70b67bc44 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:12.046261 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:08:12.088061 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d7c3b5-2b53-4b58-9eb3-11b70b67bc44 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:12.123828 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee07b87-57c3-4ff9-93a7-a690f57038d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:12.124534 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee07b87-57c3-4ff9-93a7-a690f57038d1 None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:12.124826 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee07b87-57c3-4ff9-93a7-a690f57038d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:12.125108 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee07b87-57c3-4ff9-93a7-a690f57038d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:12.125957 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee07b87-57c3-4ff9-93a7-a690f57038d1 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:08:12.127342 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee07b87-57c3-4ff9-93a7-a690f57038d1 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:08:12.211218 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 595/1169] 10.222.0.40 () {60 vars in 1138 bytes} [Mon Jul 18 06:08:11 2022] POST /identity/v3/domains => generated 267 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:08:14.278523 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb0765-1671-427d-bd54-04a041930c28 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:14.279093 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb0765-1671-427d-bd54-04a041930c28 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:14.279093 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb0765-1671-427d-bd54-04a041930c28 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:14.280494 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 575/1170] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:08:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:08:14.289339 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e80984-debe-4c10-9e0c-9fa5b924578a None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:14.289828 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e80984-debe-4c10-9e0c-9fa5b924578a None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:14.289931 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e80984-debe-4c10-9e0c-9fa5b924578a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:14.306953 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-a5e80984-debe-4c10-9e0c-9fa5b924578a None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:08:14.308159 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-a5e80984-debe-4c10-9e0c-9fa5b924578a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:08:14.374349 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 596/1171] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:08:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:08:14.398744 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb0765-1671-427d-bd54-04a041930c28 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:14.411372 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:08:14.500561 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb0765-1671-427d-bd54-04a041930c28 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:14.534861 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92123bc-4644-4b0d-a552-3b924110e5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:14.535627 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92123bc-4644-4b0d-a552-3b924110e5cf None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:14.535906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92123bc-4644-4b0d-a552-3b924110e5cf None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:14.536301 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92123bc-4644-4b0d-a552-3b924110e5cf None admin] PATH_INFO: `/v3/domains/bdd2f976b88044619bb190248a3a5789` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:14.546891 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92123bc-4644-4b0d-a552-3b924110e5cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=bdd2f976b88044619bb190248a3a5789)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:08:14.548333 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92123bc-4644-4b0d-a552-3b924110e5cf None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:08:14.549994 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 576/1172] 10.222.0.40 () {56 vars in 1183 bytes} [Mon Jul 18 06:08:14 2022] GET /identity/v3/domains/bdd2f976b88044619bb190248a3a5789 => generated 267 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:08:14.556240 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e80984-debe-4c10-9e0c-9fa5b924578a None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:14.611423 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:08:14.657732 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e80984-debe-4c10-9e0c-9fa5b924578a None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:14.693338 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72428a30-2819-4c2c-a1c8-c4a5aaad7a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:14.694121 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72428a30-2819-4c2c-a1c8-c4a5aaad7a39 None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:14.694411 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72428a30-2819-4c2c-a1c8-c4a5aaad7a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:14.694692 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72428a30-2819-4c2c-a1c8-c4a5aaad7a39 None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:14.695409 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72428a30-2819-4c2c-a1c8-c4a5aaad7a39 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:08:14.696887 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72428a30-2819-4c2c-a1c8-c4a5aaad7a39 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:08:14.794119 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 597/1173] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Jul 18 06:08:14 2022] POST /identity/v3/projects => generated 361 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:08:16.854206 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6ebba-bf9e-45c6-bdf0-ef138a692568 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:16.854757 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6ebba-bf9e-45c6-bdf0-ef138a692568 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:16.854757 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6ebba-bf9e-45c6-bdf0-ef138a692568 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:16.856118 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 577/1174] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:08:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:08:16.861296 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5842e-6487-42a4-8557-b9de94c7eb4b None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:16.861807 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5842e-6487-42a4-8557-b9de94c7eb4b None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:16.861928 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5842e-6487-42a4-8557-b9de94c7eb4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:16.878648 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-86e5842e-6487-42a4-8557-b9de94c7eb4b None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:08:16.880564 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-86e5842e-6487-42a4-8557-b9de94c7eb4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:08:16.946732 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 598/1175] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:08:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:08:16.971930 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d6ebba-bf9e-45c6-bdf0-ef138a692568 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:17.041439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:08:17.073503 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d6ebba-bf9e-45c6-bdf0-ef138a692568 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:17.105573 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d42132a-e20e-4b8a-9c04-0affe426505a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:17.106429 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42132a-e20e-4b8a-9c04-0affe426505a None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:17.106718 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42132a-e20e-4b8a-9c04-0affe426505a None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:17.106998 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42132a-e20e-4b8a-9c04-0affe426505a None admin] PATH_INFO: `/v3/domains/bdd2f976b88044619bb190248a3a5789` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:17.108173 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d42132a-e20e-4b8a-9c04-0affe426505a None admin] RBAC: Authorizing `identity:get_domain(domain_id=bdd2f976b88044619bb190248a3a5789)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:08:17.109620 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d42132a-e20e-4b8a-9c04-0affe426505a None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:08:17.111287 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 578/1176] 10.222.0.40 () {56 vars in 1183 bytes} [Mon Jul 18 06:08:16 2022] GET /identity/v3/domains/bdd2f976b88044619bb190248a3a5789 => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:08:17.117701 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e5842e-6487-42a4-8557-b9de94c7eb4b None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:17.152458 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:08:17.214948 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e5842e-6487-42a4-8557-b9de94c7eb4b None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:17.248188 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112c94af-2f7f-41a2-8937-4eb05b33e354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:17.248873 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c94af-2f7f-41a2-8937-4eb05b33e354 None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:17.249161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c94af-2f7f-41a2-8937-4eb05b33e354 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:17.249439 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c94af-2f7f-41a2-8937-4eb05b33e354 None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:17.250152 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112c94af-2f7f-41a2-8937-4eb05b33e354 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:08:17.251580 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112c94af-2f7f-41a2-8937-4eb05b33e354 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:08:17.340826 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 599/1177] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Jul 18 06:08:17 2022] POST /identity/v3/users => generated 311 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:08:19.377036 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a712df-0583-4244-a98a-81047c990209 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:19.377572 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a712df-0583-4244-a98a-81047c990209 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:19.377572 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a712df-0583-4244-a98a-81047c990209 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:19.378961 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 579/1178] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:08:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:08:19.384839 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196310e-4634-4754-b8e6-fac70f5c46b5 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:19.385281 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196310e-4634-4754-b8e6-fac70f5c46b5 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:19.385400 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196310e-4634-4754-b8e6-fac70f5c46b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:19.402774 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-e196310e-4634-4754-b8e6-fac70f5c46b5 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:08:19.403954 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e196310e-4634-4754-b8e6-fac70f5c46b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:08:19.470396 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 600/1179] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:08:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:08:19.494293 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a712df-0583-4244-a98a-81047c990209 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:19.594687 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a712df-0583-4244-a98a-81047c990209 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:19.627228 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d846676-5505-452c-b6b1-6b59a625f706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:19.627947 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d846676-5505-452c-b6b1-6b59a625f706 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:19.628236 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d846676-5505-452c-b6b1-6b59a625f706 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:19.628545 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d846676-5505-452c-b6b1-6b59a625f706 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:19.635816 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:08:19.640124 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d846676-5505-452c-b6b1-6b59a625f706 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:08:19.648274 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-7d846676-5505-452c-b6b1-6b59a625f706 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:08:19.648274 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:08:19.648274 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:08:19.648274 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:08:19.648274 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:08:19.648274 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:08:19.648274 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:08:19.648274 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:08:19.648274 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:08:19.648274 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:08:19.648274 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:08:19.648274 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:08:19.648274 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:08:19.648274 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:08:19.648274 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:08:19.648274 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:08:19.656515 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:08:19.656515 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:08:19.656515 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:08:19.656515 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:08:19.656515 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:08:19.656515 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:08:19.656515 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:08:19.656515 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:08:19.656515 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:08:19.656515 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:08:19.656515 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:08:19.656515 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:08:19.656515 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:08:19.656515 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:08:19.656515 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:08:19.656515 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:08:19.657686 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:08:19.657686 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:08:19.657686 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:08:19.657686 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:08:19.657686 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:08:19.657686 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:08:19.657686 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:08:19.657686 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:08:19.657686 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:08:19.657686 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:08:19.657686 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:08:19.657686 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:08:19.657686 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 580/1180] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 18 06:08:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:08:19.658583 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e196310e-4634-4754-b8e6-fac70f5c46b5 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:19.679745 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:08:19.759871 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e196310e-4634-4754-b8e6-fac70f5c46b5 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:19.793375 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6e772-2bb3-42e2-901f-d6715553becc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:19.794039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6e772-2bb3-42e2-901f-d6715553becc None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:19.794324 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6e772-2bb3-42e2-901f-d6715553becc None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:19.794602 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6e772-2bb3-42e2-901f-d6715553becc None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:19.795343 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a6e772-2bb3-42e2-901f-d6715553becc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:08:19.795650 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a6e772-2bb3-42e2-901f-d6715553becc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:08:19.797061 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a6e772-2bb3-42e2-901f-d6715553becc None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:08:19.805282 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 601/1181] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:08:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:08:19.811151 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d846676-5505-452c-b6b1-6b59a625f706 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:19.909908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d846676-5505-452c-b6b1-6b59a625f706 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:19.942912 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bd887a-68a2-4bc8-9d47-42d5026bb246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:19.943790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd887a-68a2-4bc8-9d47-42d5026bb246 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:19.944074 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd887a-68a2-4bc8-9d47-42d5026bb246 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:19.944352 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd887a-68a2-4bc8-9d47-42d5026bb246 None admin] PATH_INFO: `/v3/users/e1f9ffe69fe247e3ada88f3364bdbcd8` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:19.958709 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bd887a-68a2-4bc8-9d47-42d5026bb246 None admin] RBAC: Authorizing `identity:get_user(user_id=e1f9ffe69fe247e3ada88f3364bdbcd8)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:08:19.960136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bd887a-68a2-4bc8-9d47-42d5026bb246 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:08:19.961872 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 581/1182] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:08:19 2022] GET /identity/v3/users/e1f9ffe69fe247e3ada88f3364bdbcd8 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:08:19.967813 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6e772-2bb3-42e2-901f-d6715553becc None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:20.051271 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:08:20.068606 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6e772-2bb3-42e2-901f-d6715553becc None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:20.101712 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e56f6-0441-4eec-b798-394e18b640a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:20.102361 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e56f6-0441-4eec-b798-394e18b640a2 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:20.102650 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e56f6-0441-4eec-b798-394e18b640a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:20.102998 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e56f6-0441-4eec-b798-394e18b640a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:20.169003 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776e56f6-0441-4eec-b798-394e18b640a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:08:20.170388 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776e56f6-0441-4eec-b798-394e18b640a2 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:08:20.251145 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:08:20.267650 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 602/1183] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:08:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 06:08:20.273795 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bd887a-68a2-4bc8-9d47-42d5026bb246 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:20.330164 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:08:20.373425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bd887a-68a2-4bc8-9d47-42d5026bb246 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:20.406024 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296140a2-6b7d-4a95-b755-0096f0e30365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:20.406913 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296140a2-6b7d-4a95-b755-0096f0e30365 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:20.407198 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296140a2-6b7d-4a95-b755-0096f0e30365 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:20.407476 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296140a2-6b7d-4a95-b755-0096f0e30365 None admin] PATH_INFO: `/v3/projects/2acdf0a7eb9a4f86bcdfe50553008efe` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:20.408679 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296140a2-6b7d-4a95-b755-0096f0e30365 None admin] RBAC: Authorizing `identity:get_project(project_id=2acdf0a7eb9a4f86bcdfe50553008efe)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:08:20.410356 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296140a2-6b7d-4a95-b755-0096f0e30365 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:08:20.412988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 582/1184] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:08:20 2022] GET /identity/v3/projects/2acdf0a7eb9a4f86bcdfe50553008efe => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:08:20.419391 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e56f6-0441-4eec-b798-394e18b640a2 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:20.515536 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e56f6-0441-4eec-b798-394e18b640a2 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:20.548579 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3553a42-760e-4ae4-8e85-58a25292c181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:20.549229 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3553a42-760e-4ae4-8e85-58a25292c181 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:20.549513 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3553a42-760e-4ae4-8e85-58a25292c181 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:20.549790 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3553a42-760e-4ae4-8e85-58a25292c181 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:20.550643 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3553a42-760e-4ae4-8e85-58a25292c181 None admin] RBAC: Adding query filter params (role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc, scope.project.id=2acdf0a7eb9a4f86bcdfe50553008efe, user.id=e1f9ffe69fe247e3ada88f3364bdbcd8) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:08:20.550954 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3553a42-760e-4ae4-8e85-58a25292c181 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:08:20.553039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3553a42-760e-4ae4-8e85-58a25292c181 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:08:20.558407 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 603/1185] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:08:20 2022] GET /identity/v3/role_assignments?user.id=e1f9ffe69fe247e3ada88f3364bdbcd8&scope.project.id=2acdf0a7eb9a4f86bcdfe50553008efe&role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:08:22.553629 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa602a-5f94-4325-b779-db209f334ad9 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:22.554186 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa602a-5f94-4325-b779-db209f334ad9 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:22.554186 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa602a-5f94-4325-b779-db209f334ad9 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:22.555561 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 583/1186] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:08:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:08:22.560953 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a696db-3ba6-4599-9cf0-04cd20644ac5 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:22.561357 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a696db-3ba6-4599-9cf0-04cd20644ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:22.561505 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a696db-3ba6-4599-9cf0-04cd20644ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:22.578601 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-59a696db-3ba6-4599-9cf0-04cd20644ac5 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:08:22.579765 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-59a696db-3ba6-4599-9cf0-04cd20644ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:08:22.623350 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:08:22.645230 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 604/1187] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:08:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 06:08:22.668224 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa602a-5f94-4325-b779-db209f334ad9 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:22.768441 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa602a-5f94-4325-b779-db209f334ad9 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:22.792190 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:08:22.803069 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a035aa4f-2739-4197-848d-80aca879f71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:22.808491 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035aa4f-2739-4197-848d-80aca879f71d None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:22.808491 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035aa4f-2739-4197-848d-80aca879f71d None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:22.808491 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035aa4f-2739-4197-848d-80aca879f71d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:22.814490 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a035aa4f-2739-4197-848d-80aca879f71d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:08:22.822213 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-a035aa4f-2739-4197-848d-80aca879f71d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:08:22.822213 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:08:22.822213 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:08:22.822213 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:08:22.822213 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:08:22.822213 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:08:22.822213 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:08:22.822213 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:08:22.822213 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:08:22.822213 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:08:22.822213 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:08:22.822213 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:08:22.822213 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:08:22.822213 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:08:22.822213 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:08:22.822213 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:08:22.823317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:08:22.823317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:08:22.823317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:08:22.823317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:08:22.823317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:08:22.823317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:08:22.823317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:08:22.823317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:08:22.823317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:08:22.823317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:08:22.823317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:08:22.823317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:08:22.823317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:08:22.823317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:08:22.823317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:08:22.823317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:08:22.824389 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:08:22.824389 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:08:22.824389 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:08:22.824389 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:08:22.824389 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:08:22.824389 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:08:22.824389 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:08:22.824389 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:08:22.824389 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:08:22.824389 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:08:22.824389 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:08:22.824389 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:08:22.824389 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 584/1188] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 18 06:08:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:08:22.829596 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a696db-3ba6-4599-9cf0-04cd20644ac5 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:22.929491 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a696db-3ba6-4599-9cf0-04cd20644ac5 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:22.962799 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe5ffd-5c00-41df-9ac3-abc946ce300e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:22.963468 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe5ffd-5c00-41df-9ac3-abc946ce300e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:22.963757 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe5ffd-5c00-41df-9ac3-abc946ce300e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:22.964035 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe5ffd-5c00-41df-9ac3-abc946ce300e None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:22.964816 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fe5ffd-5c00-41df-9ac3-abc946ce300e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:08:22.965124 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fe5ffd-5c00-41df-9ac3-abc946ce300e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:08:22.966506 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fe5ffd-5c00-41df-9ac3-abc946ce300e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:08:22.975223 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 605/1189] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:08:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:08:22.981237 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a035aa4f-2739-4197-848d-80aca879f71d None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:23.079930 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a035aa4f-2739-4197-848d-80aca879f71d None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:23.113275 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6497bdd-a286-4a23-ba4b-cfc7aabecb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:23.114167 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6497bdd-a286-4a23-ba4b-cfc7aabecb9a None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:23.114456 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6497bdd-a286-4a23-ba4b-cfc7aabecb9a None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:23.114736 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6497bdd-a286-4a23-ba4b-cfc7aabecb9a None admin] PATH_INFO: `/v3/users/e1f9ffe69fe247e3ada88f3364bdbcd8` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:23.115916 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6497bdd-a286-4a23-ba4b-cfc7aabecb9a None admin] RBAC: Authorizing `identity:get_user(user_id=e1f9ffe69fe247e3ada88f3364bdbcd8)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:08:23.117371 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6497bdd-a286-4a23-ba4b-cfc7aabecb9a None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:08:23.118999 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 585/1190] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:08:22 2022] GET /identity/v3/users/e1f9ffe69fe247e3ada88f3364bdbcd8 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:08:23.124826 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe5ffd-5c00-41df-9ac3-abc946ce300e None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:23.167253 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:08:23.224208 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe5ffd-5c00-41df-9ac3-abc946ce300e None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:23.256071 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41da8a-c46b-43aa-ae69-8f6707316f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:23.256952 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41da8a-c46b-43aa-ae69-8f6707316f92 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:23.257237 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41da8a-c46b-43aa-ae69-8f6707316f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:23.257514 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41da8a-c46b-43aa-ae69-8f6707316f92 None admin] PATH_INFO: `/v3/projects/2acdf0a7eb9a4f86bcdfe50553008efe` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:23.258682 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e41da8a-c46b-43aa-ae69-8f6707316f92 None admin] RBAC: Authorizing `identity:get_project(project_id=2acdf0a7eb9a4f86bcdfe50553008efe)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:08:23.260117 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e41da8a-c46b-43aa-ae69-8f6707316f92 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:08:23.261859 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 606/1191] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:08:23 2022] GET /identity/v3/projects/2acdf0a7eb9a4f86bcdfe50553008efe => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:08:23.267999 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6497bdd-a286-4a23-ba4b-cfc7aabecb9a None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:23.320833 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:08:23.373054 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6497bdd-a286-4a23-ba4b-cfc7aabecb9a None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:23.407174 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13eef60-eb35-473a-9a03-99636232ebd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:23.408052 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13eef60-eb35-473a-9a03-99636232ebd5 None admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:23.408342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13eef60-eb35-473a-9a03-99636232ebd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:23.408898 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13eef60-eb35-473a-9a03-99636232ebd5 None admin] PATH_INFO: `/v3/projects/2acdf0a7eb9a4f86bcdfe50553008efe/users/e1f9ffe69fe247e3ada88f3364bdbcd8/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:23.410860 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13eef60-eb35-473a-9a03-99636232ebd5 None admin] RBAC: Authorizing `identity:create_grant(project_id=2acdf0a7eb9a4f86bcdfe50553008efe, user_id=e1f9ffe69fe247e3ada88f3364bdbcd8, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:08:23.412429 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13eef60-eb35-473a-9a03-99636232ebd5 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:08:23.488613 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 586/1192] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 18 06:08:23 2022] PUT /identity/v3/projects/2acdf0a7eb9a4f86bcdfe50553008efe/users/e1f9ffe69fe247e3ada88f3364bdbcd8/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:08:25.507922 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633eae8f-8562-46af-9b04-54005da1ecf2 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:25.508571 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633eae8f-8562-46af-9b04-54005da1ecf2 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:25.508571 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633eae8f-8562-46af-9b04-54005da1ecf2 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:25.509855 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 607/1193] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:08:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:08:25.515247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b30e35-2b55-421a-9fb3-ba16def41dfd None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:25.515673 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b30e35-2b55-421a-9fb3-ba16def41dfd None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:25.515808 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b30e35-2b55-421a-9fb3-ba16def41dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:25.533034 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-60b30e35-2b55-421a-9fb3-ba16def41dfd None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:08:25.534466 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-60b30e35-2b55-421a-9fb3-ba16def41dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:08:25.600584 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 587/1194] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:08:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:08:25.624568 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633eae8f-8562-46af-9b04-54005da1ecf2 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:25.727628 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633eae8f-8562-46af-9b04-54005da1ecf2 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:25.736405 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:08:25.769189 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508b822-7fe6-4439-85db-faee79349e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:25.769189 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508b822-7fe6-4439-85db-faee79349e83 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:25.769189 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508b822-7fe6-4439-85db-faee79349e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:25.769189 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508b822-7fe6-4439-85db-faee79349e83 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:25.772513 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f508b822-7fe6-4439-85db-faee79349e83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:08:25.779968 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-f508b822-7fe6-4439-85db-faee79349e83 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:08:25.779968 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:08:25.779968 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:08:25.779968 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:08:25.779968 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:08:25.779968 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:08:25.779968 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:08:25.779968 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:08:25.779968 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:08:25.779968 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:08:25.779968 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:08:25.779968 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:08:25.779968 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:08:25.779968 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:08:25.779968 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:08:25.779968 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:08:25.781160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:08:25.781160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:08:25.781160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:08:25.781160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:08:25.781160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:08:25.781160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:08:25.781160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:08:25.781160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:08:25.781160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:08:25.781160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:08:25.781160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:08:25.781160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:08:25.781160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:08:25.781160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:08:25.781160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:08:25.781160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:08:25.782229 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:08:25.782229 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:08:25.782229 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:08:25.782229 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:08:25.782229 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:08:25.782229 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:08:25.782229 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:08:25.782229 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:08:25.782229 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:08:25.782229 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:08:25.782229 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:08:25.782229 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:08:25.782229 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 608/1195] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 18 06:08:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:08:25.788368 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b30e35-2b55-421a-9fb3-ba16def41dfd None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:25.830764 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:08:25.885076 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b30e35-2b55-421a-9fb3-ba16def41dfd None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:25.921613 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda872f0-7d49-4050-8cbc-b2fa766c5d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:25.922268 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda872f0-7d49-4050-8cbc-b2fa766c5d31 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:25.922550 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda872f0-7d49-4050-8cbc-b2fa766c5d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:25.922824 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda872f0-7d49-4050-8cbc-b2fa766c5d31 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:25.923562 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda872f0-7d49-4050-8cbc-b2fa766c5d31 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:08:25.923860 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda872f0-7d49-4050-8cbc-b2fa766c5d31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:08:25.925236 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda872f0-7d49-4050-8cbc-b2fa766c5d31 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:08:25.933859 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 588/1196] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:08:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:08:25.939787 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508b822-7fe6-4439-85db-faee79349e83 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:26.042500 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508b822-7fe6-4439-85db-faee79349e83 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:26.076149 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b39c5-a353-4df0-8be0-f1eae215588c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:26.077039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b39c5-a353-4df0-8be0-f1eae215588c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:26.077328 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b39c5-a353-4df0-8be0-f1eae215588c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:26.077614 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b39c5-a353-4df0-8be0-f1eae215588c None admin] PATH_INFO: `/v3/users/e1f9ffe69fe247e3ada88f3364bdbcd8` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:26.078791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09b39c5-a353-4df0-8be0-f1eae215588c None admin] RBAC: Authorizing `identity:get_user(user_id=e1f9ffe69fe247e3ada88f3364bdbcd8)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:08:26.080236 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09b39c5-a353-4df0-8be0-f1eae215588c None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:08:26.081963 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 609/1197] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 18 06:08:25 2022] GET /identity/v3/users/e1f9ffe69fe247e3ada88f3364bdbcd8 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:08:26.088231 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda872f0-7d49-4050-8cbc-b2fa766c5d31 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:26.186465 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda872f0-7d49-4050-8cbc-b2fa766c5d31 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:26.188948 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:08:26.219352 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c3478a-8d0b-4925-85c7-2b023ff7df6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:26.220190 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3478a-8d0b-4925-85c7-2b023ff7df6d None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:26.220557 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3478a-8d0b-4925-85c7-2b023ff7df6d None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:26.221035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3478a-8d0b-4925-85c7-2b023ff7df6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:26.288751 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c3478a-8d0b-4925-85c7-2b023ff7df6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:08:26.290206 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c3478a-8d0b-4925-85c7-2b023ff7df6d None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:08:26.387596 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 589/1198] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:08:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 06:08:26.393476 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b39c5-a353-4df0-8be0-f1eae215588c None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:26.428674 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:08:26.491992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b39c5-a353-4df0-8be0-f1eae215588c None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:26.526078 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a8e518-471f-4f51-9c2e-5d6976508c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:26.526938 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8e518-471f-4f51-9c2e-5d6976508c17 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:26.527227 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8e518-471f-4f51-9c2e-5d6976508c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:26.527567 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8e518-471f-4f51-9c2e-5d6976508c17 None admin] PATH_INFO: `/v3/projects/2acdf0a7eb9a4f86bcdfe50553008efe` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:26.528897 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a8e518-471f-4f51-9c2e-5d6976508c17 None admin] RBAC: Authorizing `identity:get_project(project_id=2acdf0a7eb9a4f86bcdfe50553008efe)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:08:26.530340 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a8e518-471f-4f51-9c2e-5d6976508c17 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:08:26.532032 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 610/1199] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 18 06:08:26 2022] GET /identity/v3/projects/2acdf0a7eb9a4f86bcdfe50553008efe => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:08:26.538504 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c3478a-8d0b-4925-85c7-2b023ff7df6d None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:08:26.541727 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:08:26.634969 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c3478a-8d0b-4925-85c7-2b023ff7df6d None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:08:26.668667 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7945b360-8494-41d8-8f78-1664510ac094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:08:26.669253 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945b360-8494-41d8-8f78-1664510ac094 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:08:26.669538 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945b360-8494-41d8-8f78-1664510ac094 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:08:26.669819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945b360-8494-41d8-8f78-1664510ac094 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:08:26.670696 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7945b360-8494-41d8-8f78-1664510ac094 None admin] RBAC: Adding query filter params (role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc, scope.project.id=2acdf0a7eb9a4f86bcdfe50553008efe, user.id=e1f9ffe69fe247e3ada88f3364bdbcd8) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:08:26.671007 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7945b360-8494-41d8-8f78-1664510ac094 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:08:26.672438 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7945b360-8494-41d8-8f78-1664510ac094 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:08:26.678291 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 590/1200] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:08:26 2022] GET /identity/v3/role_assignments?user.id=e1f9ffe69fe247e3ada88f3364bdbcd8&scope.project.id=2acdf0a7eb9a4f86bcdfe50553008efe&role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:08:36.521946 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=48 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:08:36.675571 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=66 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:09:44.741738 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7bc23-2ca5-4a16-9163-a1f608a82625 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:44.742407 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7bc23-2ca5-4a16-9163-a1f608a82625 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:44.742407 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7bc23-2ca5-4a16-9163-a1f608a82625 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:44.743663 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 611/1201] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:09:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:09:44.749339 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7639fbe5-1d42-4455-814d-f95c179e29e1 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:44.751808 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7639fbe5-1d42-4455-814d-f95c179e29e1 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:44.751808 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7639fbe5-1d42-4455-814d-f95c179e29e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:44.768009 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-7639fbe5-1d42-4455-814d-f95c179e29e1 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:09:44.769313 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7639fbe5-1d42-4455-814d-f95c179e29e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:09:44.841399 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 591/1202] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:09:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:09:44.865805 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c7bc23-2ca5-4a16-9163-a1f608a82625 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:09:44.969348 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c7bc23-2ca5-4a16-9163-a1f608a82625 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:09:45.004712 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0947d0-4651-4085-8e49-6d863ca9b0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:09:45.005490 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0947d0-4651-4085-8e49-6d863ca9b0a1 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:45.005823 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0947d0-4651-4085-8e49-6d863ca9b0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:45.006103 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0947d0-4651-4085-8e49-6d863ca9b0a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:45.016944 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0947d0-4651-4085-8e49-6d863ca9b0a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:09:45.018345 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0947d0-4651-4085-8e49-6d863ca9b0a1 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:09:45.036487 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-1f0947d0-4651-4085-8e49-6d863ca9b0a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:09:45.036487 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:09:45.036487 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:09:45.036487 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:09:45.036487 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:09:45.036487 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:09:45.036487 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:09:45.036487 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:09:45.036487 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:09:45.036487 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:09:45.036487 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:09:45.036487 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:09:45.036487 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:09:45.036487 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:09:45.036487 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:09:45.037657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:09:45.037657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:09:45.037657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:09:45.037657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:09:45.037657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:09:45.037657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:09:45.037657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:09:45.037657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:09:45.037657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:09:45.037657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:09:45.037657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:09:45.037657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:09:45.037657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:09:45.037657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:09:45.037657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:09:45.037657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:09:45.037657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:09:45.038789 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:09:45.038789 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:09:45.038789 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:09:45.038789 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:09:45.038789 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 612/1203] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:09:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:09:45.044092 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7639fbe5-1d42-4455-814d-f95c179e29e1 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:09:45.145835 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7639fbe5-1d42-4455-814d-f95c179e29e1 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:09:45.177157 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:09:45.182786 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf61365-34aa-4f8e-8c10-bd9c7c4d6dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:09:45.183981 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf61365-34aa-4f8e-8c10-bd9c7c4d6dff None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:45.188098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf61365-34aa-4f8e-8c10-bd9c7c4d6dff None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:45.188098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf61365-34aa-4f8e-8c10-bd9c7c4d6dff None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:45.188098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf61365-34aa-4f8e-8c10-bd9c7c4d6dff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:09:45.188098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf61365-34aa-4f8e-8c10-bd9c7c4d6dff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:09:45.188098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf61365-34aa-4f8e-8c10-bd9c7c4d6dff None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:09:45.197964 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 592/1204] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:09:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:09:45.204536 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0947d0-4651-4085-8e49-6d863ca9b0a1 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:09:45.258041 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:09:45.307176 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0947d0-4651-4085-8e49-6d863ca9b0a1 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:09:45.342799 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c78775b-25ab-413b-803d-6e2f7f79ff7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:09:45.343528 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78775b-25ab-413b-803d-6e2f7f79ff7e None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:45.343835 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78775b-25ab-413b-803d-6e2f7f79ff7e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:45.344120 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78775b-25ab-413b-803d-6e2f7f79ff7e None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:45.344871 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c78775b-25ab-413b-803d-6e2f7f79ff7e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:09:45.346317 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c78775b-25ab-413b-803d-6e2f7f79ff7e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:09:45.407407 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 613/1205] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Jul 18 06:09:45 2022] POST /identity/v3/users => generated 251 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:09:47.449614 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86f55e-fc5b-479c-a512-4f4b20090625 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:47.450149 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86f55e-fc5b-479c-a512-4f4b20090625 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:47.450268 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86f55e-fc5b-479c-a512-4f4b20090625 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:47.451575 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 593/1206] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:09:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:09:47.456641 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c572f2-9d5e-4816-a82e-01753f0fae11 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:47.457055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c572f2-9d5e-4816-a82e-01753f0fae11 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:47.457196 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c572f2-9d5e-4816-a82e-01753f0fae11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:47.474738 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-f7c572f2-9d5e-4816-a82e-01753f0fae11 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:09:47.475922 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-f7c572f2-9d5e-4816-a82e-01753f0fae11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:09:47.542695 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 614/1207] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:09:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:09:47.565855 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a86f55e-fc5b-479c-a512-4f4b20090625 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:09:47.668801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a86f55e-fc5b-479c-a512-4f4b20090625 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:09:47.703228 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c35995-b5f7-4ee8-95a9-feda61997155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:09:47.703951 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c35995-b5f7-4ee8-95a9-feda61997155 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:47.704239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c35995-b5f7-4ee8-95a9-feda61997155 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:47.704553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c35995-b5f7-4ee8-95a9-feda61997155 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:47.715835 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c35995-b5f7-4ee8-95a9-feda61997155 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:09:47.723475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-26c35995-b5f7-4ee8-95a9-feda61997155 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:09:47.723475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:09:47.723475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:09:47.723475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:09:47.723475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:09:47.723475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:09:47.723475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:09:47.723475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:09:47.723475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:09:47.723475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:09:47.723475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:09:47.723475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:09:47.723475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:09:47.723475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:09:47.723475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:09:47.723475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:09:47.724919 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:09:47.724919 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:09:47.724919 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:09:47.724919 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:09:47.724919 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:09:47.724919 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:09:47.724919 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:09:47.724919 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:09:47.724919 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:09:47.724919 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:09:47.724919 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:09:47.724919 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:09:47.724919 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:09:47.724919 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:09:47.724919 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:09:47.724919 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:09:47.725906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:09:47.725906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:09:47.725906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:09:47.725906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:09:47.725906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:09:47.725906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:09:47.725906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:09:47.725906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:09:47.725906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:09:47.725906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:09:47.725906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:09:47.725906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:09:47.725906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 594/1208] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 18 06:09:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:09:47.730914 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c572f2-9d5e-4816-a82e-01753f0fae11 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:09:47.769817 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:09:47.831338 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c572f2-9d5e-4816-a82e-01753f0fae11 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:09:47.866134 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5918215a-2845-45a2-9b17-86400a9c24ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:09:47.866805 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5918215a-2845-45a2-9b17-86400a9c24ff None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:47.867096 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5918215a-2845-45a2-9b17-86400a9c24ff None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:47.867377 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5918215a-2845-45a2-9b17-86400a9c24ff None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:47.868125 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5918215a-2845-45a2-9b17-86400a9c24ff None admin] RBAC: Adding query filter params (name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:09:47.868502 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5918215a-2845-45a2-9b17-86400a9c24ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:09:47.873092 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5918215a-2845-45a2-9b17-86400a9c24ff None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:09:47.878578 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 615/1209] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 18 06:09:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:09:47.884322 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c35995-b5f7-4ee8-95a9-feda61997155 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:09:47.935672 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:09:47.984171 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c35995-b5f7-4ee8-95a9-feda61997155 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:09:48.019890 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3579c32-846e-4313-9d3a-46d378a3bf25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:09:48.020702 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3579c32-846e-4313-9d3a-46d378a3bf25 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:48.020987 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3579c32-846e-4313-9d3a-46d378a3bf25 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:48.021257 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3579c32-846e-4313-9d3a-46d378a3bf25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:48.031735 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3579c32-846e-4313-9d3a-46d378a3bf25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:09:48.033328 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3579c32-846e-4313-9d3a-46d378a3bf25 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:09:48.050823 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-c3579c32-846e-4313-9d3a-46d378a3bf25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:09:48.050823 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:09:48.050823 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:09:48.050823 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:09:48.050823 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:09:48.050823 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:09:48.050823 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:09:48.050823 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:09:48.050823 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:09:48.050823 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:09:48.050823 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:09:48.050823 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:09:48.050823 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:09:48.050823 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:09:48.050823 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:09:48.051970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:09:48.051970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:09:48.051970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:09:48.051970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:09:48.051970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:09:48.051970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:09:48.051970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:09:48.051970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:09:48.051970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:09:48.051970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:09:48.051970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:09:48.051970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:09:48.051970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:09:48.051970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:09:48.051970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:09:48.051970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:09:48.051970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:09:48.052937 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:09:48.052937 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:09:48.052937 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:09:48.052937 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:09:48.052937 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 595/1210] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:09:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:09:48.057966 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5918215a-2845-45a2-9b17-86400a9c24ff None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:09:48.157268 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:09:48.160222 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5918215a-2845-45a2-9b17-86400a9c24ff None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:09:48.193563 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c92a5-09ca-47f8-a1b3-fcfaced3f166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:09:48.194224 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c92a5-09ca-47f8-a1b3-fcfaced3f166 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:48.194509 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c92a5-09ca-47f8-a1b3-fcfaced3f166 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:48.194783 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c92a5-09ca-47f8-a1b3-fcfaced3f166 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:48.195499 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536c92a5-09ca-47f8-a1b3-fcfaced3f166 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:09:48.195805 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536c92a5-09ca-47f8-a1b3-fcfaced3f166 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:09:48.197218 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536c92a5-09ca-47f8-a1b3-fcfaced3f166 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:09:48.209425 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 616/1211] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:09:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:09:48.215190 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3579c32-846e-4313-9d3a-46d378a3bf25 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:09:48.315235 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3579c32-846e-4313-9d3a-46d378a3bf25 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:09:48.316682 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:09:48.349136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eae001-dbdf-43de-9a8c-d92d18ac7bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:09:48.350633 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eae001-dbdf-43de-9a8c-d92d18ac7bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:48.350633 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eae001-dbdf-43de-9a8c-d92d18ac7bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:48.350633 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eae001-dbdf-43de-9a8c-d92d18ac7bc8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:48.358269 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eae001-dbdf-43de-9a8c-d92d18ac7bc8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:09:48.359744 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eae001-dbdf-43de-9a8c-d92d18ac7bc8 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:09:48.375680 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-d6eae001-dbdf-43de-9a8c-d92d18ac7bc8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 06:09:48.375680 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:09:48.375680 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:09:48.375680 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:09:48.375680 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:09:48.375680 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:09:48.375680 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:09:48.375680 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:09:48.375680 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:09:48.375680 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:09:48.375680 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:09:48.375680 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:09:48.375680 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 06:09:48.375680 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 06:09:48.375680 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 06:09:48.375680 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 06:09:48.380229 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:09:48.380229 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:09:48.380229 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 06:09:48.380229 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:09:48.380229 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 06:09:48.380229 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:09:48.380229 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:09:48.380229 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:09:48.380229 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:09:48.380229 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:09:48.380229 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:09:48.380229 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:09:48.380229 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:09:48.380229 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:09:48.380229 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:09:48.380229 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:09:48.380229 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:09:48.380229 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:09:48.381551 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:09:48.381551 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:09:48.381551 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 06:09:48.381551 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 06:09:48.381551 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 06:09:48.381551 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 06:09:48.381551 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 06:09:48.381551 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 06:09:48.381551 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 06:09:48.381551 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:09:48.381551 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 596/1212] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:09:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:09:48.382969 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c92a5-09ca-47f8-a1b3-fcfaced3f166 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:09:48.484681 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c92a5-09ca-47f8-a1b3-fcfaced3f166 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:09:48.519536 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd99d0-9bc6-4be1-9795-6995b25d3e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:09:48.520202 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd99d0-9bc6-4be1-9795-6995b25d3e5f None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:48.520524 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd99d0-9bc6-4be1-9795-6995b25d3e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:48.520814 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd99d0-9bc6-4be1-9795-6995b25d3e5f None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:48.521656 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcd99d0-9bc6-4be1-9795-6995b25d3e5f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:09:48.521969 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcd99d0-9bc6-4be1-9795-6995b25d3e5f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:09:48.523448 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcd99d0-9bc6-4be1-9795-6995b25d3e5f None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:09:48.542176 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 617/1213] 10.222.0.40 () {56 vars in 1143 bytes} [Mon Jul 18 06:09:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:09:48.548792 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eae001-dbdf-43de-9a8c-d92d18ac7bc8 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:09:48.648258 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eae001-dbdf-43de-9a8c-d92d18ac7bc8 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:09:48.684783 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab74a6f1-62c3-4d90-b646-042b6598ea47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:09:48.685441 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74a6f1-62c3-4d90-b646-042b6598ea47 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:48.685779 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74a6f1-62c3-4d90-b646-042b6598ea47 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:48.686010 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74a6f1-62c3-4d90-b646-042b6598ea47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:48.716379 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:09:48.753519 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab74a6f1-62c3-4d90-b646-042b6598ea47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:09:48.754954 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab74a6f1-62c3-4d90-b646-042b6598ea47 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:09:48.853093 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 597/1214] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:09:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 06:09:48.859713 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd99d0-9bc6-4be1-9795-6995b25d3e5f None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:09:48.878864 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:09:48.957664 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd99d0-9bc6-4be1-9795-6995b25d3e5f None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:09:48.990726 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb62be2-e439-4cda-b8e9-15d6ec892864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:09:48.991508 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb62be2-e439-4cda-b8e9-15d6ec892864 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:48.991786 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb62be2-e439-4cda-b8e9-15d6ec892864 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:48.992054 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb62be2-e439-4cda-b8e9-15d6ec892864 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:49.002600 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb62be2-e439-4cda-b8e9-15d6ec892864 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:09:49.004114 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb62be2-e439-4cda-b8e9-15d6ec892864 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:09:49.022346 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-2bb62be2-e439-4cda-b8e9-15d6ec892864 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:09:49.022346 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:09:49.022346 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:09:49.022346 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:09:49.022346 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:09:49.022346 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:09:49.022346 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:09:49.022346 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:09:49.022346 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:09:49.022346 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:09:49.022346 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:09:49.022346 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:09:49.022346 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:09:49.022346 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:09:49.022346 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:09:49.023436 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:09:49.023436 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:09:49.023436 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:09:49.023436 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:09:49.023436 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:09:49.023436 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:09:49.023436 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:09:49.023436 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:09:49.023436 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:09:49.023436 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:09:49.023436 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:09:49.023436 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:09:49.023436 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:09:49.023436 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:09:49.023436 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:09:49.023436 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:09:49.023436 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:09:49.024652 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:09:49.024652 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:09:49.024652 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:09:49.024652 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:09:49.024652 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 618/1215] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:09:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:09:49.030102 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab74a6f1-62c3-4d90-b646-042b6598ea47 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:09:49.090798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:09:49.126753 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab74a6f1-62c3-4d90-b646-042b6598ea47 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:09:49.162601 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc130b-73e4-4e9c-ae42-c6612380c9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:09:49.163258 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc130b-73e4-4e9c-ae42-c6612380c9b2 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:49.163539 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc130b-73e4-4e9c-ae42-c6612380c9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:49.163814 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc130b-73e4-4e9c-ae42-c6612380c9b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:49.164566 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc130b-73e4-4e9c-ae42-c6612380c9b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:09:49.164871 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc130b-73e4-4e9c-ae42-c6612380c9b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:09:49.166303 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc130b-73e4-4e9c-ae42-c6612380c9b2 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:09:49.177988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 598/1216] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:09:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:09:49.183951 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb62be2-e439-4cda-b8e9-15d6ec892864 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:09:49.251956 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:09:49.283581 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb62be2-e439-4cda-b8e9-15d6ec892864 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:09:49.317521 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb899648-b654-4330-ae52-e6e137196c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:09:49.318311 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb899648-b654-4330-ae52-e6e137196c46 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:49.318596 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb899648-b654-4330-ae52-e6e137196c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:49.318870 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb899648-b654-4330-ae52-e6e137196c46 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:49.325972 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb899648-b654-4330-ae52-e6e137196c46 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:09:49.327409 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb899648-b654-4330-ae52-e6e137196c46 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:09:49.341827 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-bb899648-b654-4330-ae52-e6e137196c46 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:09:49.341827 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:09:49.341827 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:09:49.341827 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:09:49.341827 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:09:49.341827 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:09:49.341827 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:09:49.341827 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:09:49.341827 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:09:49.341827 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:09:49.341827 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:09:49.341827 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:09:49.341827 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 06:09:49.341827 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 06:09:49.341827 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 06:09:49.342917 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 06:09:49.342917 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:09:49.342917 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:09:49.342917 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:09:49.342917 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:09:49.342917 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:09:49.342917 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:09:49.342917 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:09:49.342917 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:09:49.342917 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:09:49.342917 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:09:49.342917 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:09:49.342917 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:09:49.342917 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:09:49.342917 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:09:49.342917 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:09:49.342917 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:09:49.344041 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 06:09:49.344041 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 06:09:49.344041 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 06:09:49.344041 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 06:09:49.344041 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 06:09:49.344041 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 06:09:49.344041 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:09:49.344041 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:09:49.344041 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 619/1217] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:09:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:09:49.349260 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc130b-73e4-4e9c-ae42-c6612380c9b2 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:09:49.453443 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc130b-73e4-4e9c-ae42-c6612380c9b2 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:09:49.475791 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:09:49.487876 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd8511-5bb8-49d9-9740-cce5922a4ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:09:49.489863 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd8511-5bb8-49d9-9740-cce5922a4ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:49.489863 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd8511-5bb8-49d9-9740-cce5922a4ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:49.489863 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd8511-5bb8-49d9-9740-cce5922a4ed3 None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:49.490136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd8511-5bb8-49d9-9740-cce5922a4ed3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:09:49.490231 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd8511-5bb8-49d9-9740-cce5922a4ed3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:09:49.491703 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd8511-5bb8-49d9-9740-cce5922a4ed3 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:09:49.504677 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 599/1218] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 18 06:09:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:09:49.510766 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb899648-b654-4330-ae52-e6e137196c46 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:09:49.618244 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb899648-b654-4330-ae52-e6e137196c46 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:09:49.651975 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:09:49.658275 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20b4ab-71cc-48be-9472-a2131f97957d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:09:49.658941 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20b4ab-71cc-48be-9472-a2131f97957d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:49.659229 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20b4ab-71cc-48be-9472-a2131f97957d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:49.659514 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20b4ab-71cc-48be-9472-a2131f97957d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:49.660349 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20b4ab-71cc-48be-9472-a2131f97957d None admin] RBAC: Adding query filter params (role.id=ab10744ff77646eab23a769ab89c74aa, scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a, user.id=a3560534427544cdabfd0ba702a0a165) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:09:49.660695 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20b4ab-71cc-48be-9472-a2131f97957d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:09:49.662188 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20b4ab-71cc-48be-9472-a2131f97957d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:09:49.667814 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 620/1219] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:09:49 2022] GET /identity/v3/role_assignments?user.id=a3560534427544cdabfd0ba702a0a165&scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a&role.id=ab10744ff77646eab23a769ab89c74aa => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:09:51.691210 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90acc6-2f30-4726-826a-2e1af2246fa2 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:51.691603 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90acc6-2f30-4726-826a-2e1af2246fa2 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:51.697300 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90acc6-2f30-4726-826a-2e1af2246fa2 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:51.697300 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 600/1220] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:09:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:09:51.698269 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52411120-1710-4aa7-b703-121867112c27 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:51.698753 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52411120-1710-4aa7-b703-121867112c27 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:51.699040 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52411120-1710-4aa7-b703-121867112c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:51.716342 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-52411120-1710-4aa7-b703-121867112c27 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:09:51.717554 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-52411120-1710-4aa7-b703-121867112c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:09:51.783506 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 621/1221] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:09:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:09:51.806841 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c90acc6-2f30-4726-826a-2e1af2246fa2 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:09:51.907973 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c90acc6-2f30-4726-826a-2e1af2246fa2 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:09:51.941754 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5874abdf-3701-4950-888c-b54157a9f98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:09:51.942482 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874abdf-3701-4950-888c-b54157a9f98f None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:51.942771 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874abdf-3701-4950-888c-b54157a9f98f None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:51.943053 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874abdf-3701-4950-888c-b54157a9f98f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:51.953479 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5874abdf-3701-4950-888c-b54157a9f98f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:09:51.961334 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-5874abdf-3701-4950-888c-b54157a9f98f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:09:51.961334 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:09:51.961334 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:09:51.961334 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:09:51.961334 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:09:51.961334 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:09:51.961334 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:09:51.961334 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:09:51.961334 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:09:51.961334 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:09:51.961334 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:09:51.961334 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:09:51.961334 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:09:51.961334 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:09:51.961334 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:09:51.961334 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:09:51.965247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:09:51.965247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:09:51.965247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:09:51.965247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:09:51.965247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:09:51.965247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:09:51.965247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:09:51.965247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:09:51.965247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:09:51.965247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:09:51.965247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:09:51.965247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:09:51.965247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:09:51.965247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:09:51.965247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:09:51.965247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:09:51.966320 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:09:51.966320 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:09:51.966320 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:09:51.966320 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:09:51.966320 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:09:51.966320 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:09:51.966320 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:09:51.966320 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:09:51.966320 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:09:51.966320 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:09:51.966320 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:09:51.966320 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:09:51.966320 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 601/1222] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 18 06:09:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:09:51.969306 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52411120-1710-4aa7-b703-121867112c27 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:09:52.068907 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52411120-1710-4aa7-b703-121867112c27 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:09:52.098908 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:09:52.109482 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db9e6c-6fac-4747-be90-5b38b7f8c6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:09:52.110148 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db9e6c-6fac-4747-be90-5b38b7f8c6ea None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:52.110434 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db9e6c-6fac-4747-be90-5b38b7f8c6ea None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:52.110739 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db9e6c-6fac-4747-be90-5b38b7f8c6ea None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:52.111488 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18db9e6c-6fac-4747-be90-5b38b7f8c6ea None admin] RBAC: Adding query filter params (name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:09:52.111828 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18db9e6c-6fac-4747-be90-5b38b7f8c6ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:09:52.113228 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18db9e6c-6fac-4747-be90-5b38b7f8c6ea None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:09:52.121161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 622/1223] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 18 06:09:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:09:52.127056 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5874abdf-3701-4950-888c-b54157a9f98f None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:09:52.172218 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:09:52.230236 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5874abdf-3701-4950-888c-b54157a9f98f None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:09:52.262909 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c2717-2c43-4de9-881d-8d0cfccd7354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:09:52.264995 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c2717-2c43-4de9-881d-8d0cfccd7354 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:52.264995 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c2717-2c43-4de9-881d-8d0cfccd7354 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:52.264995 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c2717-2c43-4de9-881d-8d0cfccd7354 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:52.275047 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5c2717-2c43-4de9-881d-8d0cfccd7354 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:09:52.276556 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5c2717-2c43-4de9-881d-8d0cfccd7354 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:09:52.297641 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-3b5c2717-2c43-4de9-881d-8d0cfccd7354 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:09:52.297641 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:09:52.297641 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:09:52.297641 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:09:52.297641 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:09:52.297641 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:09:52.297641 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:09:52.297641 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:09:52.297641 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:09:52.297641 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:09:52.297641 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:09:52.297641 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:09:52.297641 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:09:52.297641 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:09:52.297641 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:09:52.298798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:09:52.298798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:09:52.298798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:09:52.298798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:09:52.298798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:09:52.298798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:09:52.298798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:09:52.298798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:09:52.298798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:09:52.298798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:09:52.298798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:09:52.298798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:09:52.298798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:09:52.298798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:09:52.298798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:09:52.298798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:09:52.298798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:09:52.300103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:09:52.300103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:09:52.300103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:09:52.300103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:09:52.300103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 602/1224] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:09:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:09:52.304658 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db9e6c-6fac-4747-be90-5b38b7f8c6ea None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:09:52.405267 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db9e6c-6fac-4747-be90-5b38b7f8c6ea None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:09:52.439366 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4496a03-8a6e-4012-a794-0740bd73f2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:09:52.440029 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4496a03-8a6e-4012-a794-0740bd73f2fb None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:52.440320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4496a03-8a6e-4012-a794-0740bd73f2fb None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:52.440638 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4496a03-8a6e-4012-a794-0740bd73f2fb None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:52.441362 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4496a03-8a6e-4012-a794-0740bd73f2fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:09:52.441674 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4496a03-8a6e-4012-a794-0740bd73f2fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:09:52.443053 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4496a03-8a6e-4012-a794-0740bd73f2fb None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:09:52.455412 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 623/1225] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:09:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:09:52.461108 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c2717-2c43-4de9-881d-8d0cfccd7354 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:09:52.554937 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:09:52.561977 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c2717-2c43-4de9-881d-8d0cfccd7354 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:09:52.594771 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0bf2e2-67e7-4a38-add6-97d51328f261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:09:52.595566 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0bf2e2-67e7-4a38-add6-97d51328f261 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:52.595851 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0bf2e2-67e7-4a38-add6-97d51328f261 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:52.596188 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0bf2e2-67e7-4a38-add6-97d51328f261 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:52.603571 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0bf2e2-67e7-4a38-add6-97d51328f261 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:09:52.605109 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0bf2e2-67e7-4a38-add6-97d51328f261 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:09:52.619107 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-0f0bf2e2-67e7-4a38-add6-97d51328f261 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 06:09:52.619107 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:09:52.619107 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:09:52.619107 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:09:52.619107 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:09:52.619107 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:09:52.619107 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:09:52.619107 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:09:52.619107 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:09:52.619107 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:09:52.619107 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:09:52.619107 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:09:52.619107 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 06:09:52.619107 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 06:09:52.619107 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 06:09:52.619107 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 06:09:52.620270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:09:52.620270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:09:52.620270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 06:09:52.620270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:09:52.620270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 06:09:52.620270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:09:52.620270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:09:52.620270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:09:52.620270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:09:52.620270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:09:52.620270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:09:52.620270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:09:52.620270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:09:52.620270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:09:52.620270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:09:52.620270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:09:52.620270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:09:52.620270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:09:52.621850 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:09:52.621850 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:09:52.621850 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 06:09:52.621850 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 06:09:52.621850 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 06:09:52.621850 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 06:09:52.621850 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 06:09:52.621850 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 06:09:52.621850 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 06:09:52.621850 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:09:52.621850 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 603/1226] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:09:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:09:52.626971 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4496a03-8a6e-4012-a794-0740bd73f2fb None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:09:52.675627 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:09:52.728767 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4496a03-8a6e-4012-a794-0740bd73f2fb None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:09:52.764219 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35c1e5e-7e9d-4081-9fba-1f2c6d01ea51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:09:52.764921 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c1e5e-7e9d-4081-9fba-1f2c6d01ea51 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:52.765209 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c1e5e-7e9d-4081-9fba-1f2c6d01ea51 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:52.765487 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c1e5e-7e9d-4081-9fba-1f2c6d01ea51 None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:52.766326 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35c1e5e-7e9d-4081-9fba-1f2c6d01ea51 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:09:52.766638 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35c1e5e-7e9d-4081-9fba-1f2c6d01ea51 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:09:52.768075 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35c1e5e-7e9d-4081-9fba-1f2c6d01ea51 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:09:52.785510 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 624/1227] 10.222.0.40 () {56 vars in 1143 bytes} [Mon Jul 18 06:09:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:09:52.791225 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0bf2e2-67e7-4a38-add6-97d51328f261 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:09:52.891714 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0bf2e2-67e7-4a38-add6-97d51328f261 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:09:52.925582 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa37157-8509-4f02-9806-fec41724549b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:09:52.926368 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa37157-8509-4f02-9806-fec41724549b None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:52.926655 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa37157-8509-4f02-9806-fec41724549b None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:52.926934 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa37157-8509-4f02-9806-fec41724549b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:52.938065 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa37157-8509-4f02-9806-fec41724549b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:09:52.939533 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa37157-8509-4f02-9806-fec41724549b None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:09:52.956308 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-7aa37157-8509-4f02-9806-fec41724549b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:09:52.956308 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:09:52.956308 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:09:52.956308 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:09:52.956308 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:09:52.956308 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:09:52.956308 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:09:52.956308 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:09:52.956308 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:09:52.956308 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:09:52.956308 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:09:52.956308 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:09:52.956308 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:09:52.956308 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:09:52.956308 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:09:52.957397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:09:52.957397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:09:52.957397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:09:52.957397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:09:52.957397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:09:52.957397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:09:52.957397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:09:52.957397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:09:52.957397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:09:52.957397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:09:52.957397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:09:52.957397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:09:52.957397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:09:52.957397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:09:52.957397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:09:52.957397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:09:52.957397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:09:52.958476 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:09:52.958476 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:09:52.958476 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:09:52.958476 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:09:52.958476 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 604/1228] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:09:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:09:52.963673 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35c1e5e-7e9d-4081-9fba-1f2c6d01ea51 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:09:53.066101 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35c1e5e-7e9d-4081-9fba-1f2c6d01ea51 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:09:53.068427 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:09:53.099211 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7249145-bcdf-4f8f-a77f-d177a976d075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:09:53.099870 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7249145-bcdf-4f8f-a77f-d177a976d075 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:53.100180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7249145-bcdf-4f8f-a77f-d177a976d075 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:53.100533 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7249145-bcdf-4f8f-a77f-d177a976d075 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:53.101279 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7249145-bcdf-4f8f-a77f-d177a976d075 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:09:53.101594 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7249145-bcdf-4f8f-a77f-d177a976d075 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:09:53.102971 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7249145-bcdf-4f8f-a77f-d177a976d075 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:09:53.114882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 625/1229] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:09:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:09:53.121919 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa37157-8509-4f02-9806-fec41724549b None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:09:53.131528 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:09:53.220164 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa37157-8509-4f02-9806-fec41724549b None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:09:53.254078 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf3896e-d8b4-4209-b6be-c5459dcb99d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:09:53.254870 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3896e-d8b4-4209-b6be-c5459dcb99d8 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:53.255154 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3896e-d8b4-4209-b6be-c5459dcb99d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:53.255432 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3896e-d8b4-4209-b6be-c5459dcb99d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:53.262382 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf3896e-d8b4-4209-b6be-c5459dcb99d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:09:53.263813 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf3896e-d8b4-4209-b6be-c5459dcb99d8 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:09:53.278446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-6bf3896e-d8b4-4209-b6be-c5459dcb99d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:09:53.278446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:09:53.278446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:09:53.278446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:09:53.278446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:09:53.278446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:09:53.278446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:09:53.278446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:09:53.278446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:09:53.278446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:09:53.278446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:09:53.278446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:09:53.278446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 06:09:53.278446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 06:09:53.278446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 06:09:53.279410 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 06:09:53.279410 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:09:53.279410 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:09:53.279410 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:09:53.279410 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:09:53.279410 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:09:53.279410 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:09:53.279410 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:09:53.279410 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:09:53.279410 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:09:53.279410 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:09:53.279410 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:09:53.279410 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:09:53.279410 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:09:53.279410 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:09:53.279410 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:09:53.279410 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:09:53.280374 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 06:09:53.280374 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 06:09:53.280374 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 06:09:53.280374 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 06:09:53.280374 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 06:09:53.280374 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 06:09:53.280374 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:09:53.280374 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:09:53.280374 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 605/1230] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:09:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:09:53.286048 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7249145-bcdf-4f8f-a77f-d177a976d075 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:09:53.388043 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7249145-bcdf-4f8f-a77f-d177a976d075 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:09:53.422533 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881041fe-a345-44cf-bdb8-ead0a6844dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:09:53.423194 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881041fe-a345-44cf-bdb8-ead0a6844dbc None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:53.423484 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881041fe-a345-44cf-bdb8-ead0a6844dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:53.423763 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881041fe-a345-44cf-bdb8-ead0a6844dbc None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:53.424575 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881041fe-a345-44cf-bdb8-ead0a6844dbc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:09:53.424890 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881041fe-a345-44cf-bdb8-ead0a6844dbc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:09:53.426318 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881041fe-a345-44cf-bdb8-ead0a6844dbc None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:09:53.438397 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 626/1231] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 18 06:09:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:09:53.444820 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf3896e-d8b4-4209-b6be-c5459dcb99d8 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:09:53.516328 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:09:53.550470 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf3896e-d8b4-4209-b6be-c5459dcb99d8 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:09:53.584193 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b337f49-4467-4562-acb2-0b11f08be6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:09:53.585102 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b337f49-4467-4562-acb2-0b11f08be6bc None admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:53.585384 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b337f49-4467-4562-acb2-0b11f08be6bc None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:53.585659 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b337f49-4467-4562-acb2-0b11f08be6bc None admin] PATH_INFO: `/v3/projects/8ed50a8a3e184bde9269d7b0d9573d4a/users/a3560534427544cdabfd0ba702a0a165/roles/ab10744ff77646eab23a769ab89c74aa` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:53.600144 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b337f49-4467-4562-acb2-0b11f08be6bc None admin] RBAC: Authorizing `identity:create_grant(project_id=8ed50a8a3e184bde9269d7b0d9573d4a, user_id=a3560534427544cdabfd0ba702a0a165, role_id=ab10744ff77646eab23a769ab89c74aa)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:09:53.601597 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b337f49-4467-4562-acb2-0b11f08be6bc None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:09:53.759602 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 606/1232] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 18 06:09:53 2022] PUT /identity/v3/projects/8ed50a8a3e184bde9269d7b0d9573d4a/users/a3560534427544cdabfd0ba702a0a165/roles/ab10744ff77646eab23a769ab89c74aa => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:09:55.798779 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58849b4a-173a-4022-a6ce-262667514724 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:55.799300 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58849b4a-173a-4022-a6ce-262667514724 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:55.799391 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58849b4a-173a-4022-a6ce-262667514724 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:55.800739 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 627/1233] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:09:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:09:55.806065 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9dd39-d77d-404a-8bd0-64b8ba9fdf73 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:55.806506 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9dd39-d77d-404a-8bd0-64b8ba9fdf73 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:55.806629 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9dd39-d77d-404a-8bd0-64b8ba9fdf73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:55.822936 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-8da9dd39-d77d-404a-8bd0-64b8ba9fdf73 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:09:55.824686 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8da9dd39-d77d-404a-8bd0-64b8ba9fdf73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:09:55.890788 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 607/1234] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:09:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:09:55.914489 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58849b4a-173a-4022-a6ce-262667514724 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:09:55.940152 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:09:56.017514 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58849b4a-173a-4022-a6ce-262667514724 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:09:56.050261 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f66cf4-4bfb-459e-a6e0-c5ea86798c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:09:56.051047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f66cf4-4bfb-459e-a6e0-c5ea86798c4f None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:56.051328 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f66cf4-4bfb-459e-a6e0-c5ea86798c4f None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:56.051600 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f66cf4-4bfb-459e-a6e0-c5ea86798c4f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:56.061944 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f66cf4-4bfb-459e-a6e0-c5ea86798c4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:09:56.069777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-d3f66cf4-4bfb-459e-a6e0-c5ea86798c4f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:09:56.069777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:09:56.069777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:09:56.069777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:09:56.069777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:09:56.069777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:09:56.069777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:09:56.069777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:09:56.069777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:09:56.069777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:09:56.069777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:09:56.069777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:09:56.069777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:09:56.069777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:09:56.069777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:09:56.069777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:09:56.070893 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:09:56.070893 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:09:56.070893 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:09:56.070893 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:09:56.070893 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:09:56.070893 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:09:56.070893 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:09:56.070893 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:09:56.070893 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:09:56.070893 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:09:56.070893 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:09:56.070893 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:09:56.070893 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:09:56.070893 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:09:56.070893 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:09:56.070893 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:09:56.071896 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:09:56.071896 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:09:56.071896 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:09:56.071896 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:09:56.071896 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:09:56.071896 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:09:56.071896 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:09:56.071896 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:09:56.071896 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:09:56.071896 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:09:56.071896 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 06:09:56.071896 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:09:56.071896 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 628/1235] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 18 06:09:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:09:56.077714 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da9dd39-d77d-404a-8bd0-64b8ba9fdf73 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:09:56.123294 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:09:56.174117 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da9dd39-d77d-404a-8bd0-64b8ba9fdf73 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:09:56.207789 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063d13fa-0eb0-4885-9678-5dc03ca435f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:09:56.208377 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d13fa-0eb0-4885-9678-5dc03ca435f1 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:56.208707 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d13fa-0eb0-4885-9678-5dc03ca435f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:56.208980 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d13fa-0eb0-4885-9678-5dc03ca435f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:56.209724 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063d13fa-0eb0-4885-9678-5dc03ca435f1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:09:56.210031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063d13fa-0eb0-4885-9678-5dc03ca435f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:09:56.211441 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063d13fa-0eb0-4885-9678-5dc03ca435f1 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:09:56.220024 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 608/1236] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 18 06:09:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:09:56.225803 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f66cf4-4bfb-459e-a6e0-c5ea86798c4f None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:09:56.310811 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:09:56.327152 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f66cf4-4bfb-459e-a6e0-c5ea86798c4f None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:09:56.359919 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38218d-8b1c-4494-840e-2d6e506dab6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:09:56.360770 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38218d-8b1c-4494-840e-2d6e506dab6a None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:56.361427 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38218d-8b1c-4494-840e-2d6e506dab6a None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:56.361704 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38218d-8b1c-4494-840e-2d6e506dab6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:56.372942 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38218d-8b1c-4494-840e-2d6e506dab6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:09:56.374404 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38218d-8b1c-4494-840e-2d6e506dab6a None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:09:56.392018 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-ea38218d-8b1c-4494-840e-2d6e506dab6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:09:56.392018 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:09:56.392018 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:09:56.392018 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:09:56.392018 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:09:56.392018 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:09:56.392018 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:09:56.392018 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:09:56.392018 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:09:56.392018 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:09:56.392018 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:09:56.392018 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:09:56.392018 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:09:56.392018 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:09:56.392018 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:09:56.393221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:09:56.393221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:09:56.393221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:09:56.393221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:09:56.393221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:09:56.393221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:09:56.393221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:09:56.393221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:09:56.393221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:09:56.393221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:09:56.393221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:09:56.393221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:09:56.393221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:09:56.393221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:09:56.393221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:09:56.393221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:09:56.393221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:09:56.394156 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:09:56.394156 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:09:56.394156 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:09:56.394156 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:09:56.394156 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 629/1237] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:09:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:09:56.399411 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063d13fa-0eb0-4885-9678-5dc03ca435f1 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:09:56.498765 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063d13fa-0eb0-4885-9678-5dc03ca435f1 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:09:56.507877 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:09:56.532766 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de8ff8-3c0c-4ac4-952e-a5b5d29ae643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:09:56.533428 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de8ff8-3c0c-4ac4-952e-a5b5d29ae643 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:56.533716 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de8ff8-3c0c-4ac4-952e-a5b5d29ae643 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:56.533995 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de8ff8-3c0c-4ac4-952e-a5b5d29ae643 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:56.534717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20de8ff8-3c0c-4ac4-952e-a5b5d29ae643 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:09:56.535021 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20de8ff8-3c0c-4ac4-952e-a5b5d29ae643 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:09:56.536375 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20de8ff8-3c0c-4ac4-952e-a5b5d29ae643 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:09:56.548465 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 609/1238] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:09:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:09:56.554491 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38218d-8b1c-4494-840e-2d6e506dab6a None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:09:56.655181 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38218d-8b1c-4494-840e-2d6e506dab6a None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:09:56.691116 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b860873-0913-4548-b1db-074eb0c22a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:09:56.691932 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b860873-0913-4548-b1db-074eb0c22a6f None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:56.692221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b860873-0913-4548-b1db-074eb0c22a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:56.692537 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b860873-0913-4548-b1db-074eb0c22a6f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:56.695408 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:09:56.701633 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b860873-0913-4548-b1db-074eb0c22a6f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:09:56.703112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b860873-0913-4548-b1db-074eb0c22a6f None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:09:56.718070 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-4b860873-0913-4548-b1db-074eb0c22a6f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 06:09:56.718070 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:09:56.718070 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:09:56.718070 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:09:56.718070 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:09:56.718070 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:09:56.718070 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:09:56.718070 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:09:56.718070 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:09:56.718070 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:09:56.718070 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:09:56.718070 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:09:56.718070 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 06:09:56.718070 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 06:09:56.718070 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 06:09:56.718070 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 06:09:56.719027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:09:56.719027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:09:56.719027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 06:09:56.719027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:09:56.719027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 06:09:56.719027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:09:56.719027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:09:56.719027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:09:56.719027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:09:56.719027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:09:56.719027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:09:56.719027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:09:56.719027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:09:56.719027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:09:56.719027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:09:56.719027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:09:56.719027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:09:56.719027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:09:56.720044 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:09:56.720044 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:09:56.720044 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 06:09:56.720044 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 06:09:56.720044 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 06:09:56.720044 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 06:09:56.720044 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 06:09:56.720044 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 06:09:56.720044 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 06:09:56.720044 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:09:56.720044 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 630/1239] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:09:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:09:56.725449 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de8ff8-3c0c-4ac4-952e-a5b5d29ae643 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:09:56.832218 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de8ff8-3c0c-4ac4-952e-a5b5d29ae643 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:09:56.865952 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bac304-7eca-438c-88c9-952990e3e092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:09:56.866625 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bac304-7eca-438c-88c9-952990e3e092 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:56.866946 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bac304-7eca-438c-88c9-952990e3e092 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:56.867241 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bac304-7eca-438c-88c9-952990e3e092 None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:56.868154 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bac304-7eca-438c-88c9-952990e3e092 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:09:56.868498 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bac304-7eca-438c-88c9-952990e3e092 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:09:56.869923 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bac304-7eca-438c-88c9-952990e3e092 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:09:56.888134 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 610/1240] 10.222.0.40 () {56 vars in 1143 bytes} [Mon Jul 18 06:09:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:09:56.894004 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b860873-0913-4548-b1db-074eb0c22a6f None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:09:56.996608 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b860873-0913-4548-b1db-074eb0c22a6f None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:09:57.031438 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac87dd-8b2d-45d1-bb68-4bfd66c82885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:09:57.032091 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac87dd-8b2d-45d1-bb68-4bfd66c82885 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:57.032382 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac87dd-8b2d-45d1-bb68-4bfd66c82885 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:57.032708 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac87dd-8b2d-45d1-bb68-4bfd66c82885 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:57.094720 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:09:57.101221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac87dd-8b2d-45d1-bb68-4bfd66c82885 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:09:57.102682 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac87dd-8b2d-45d1-bb68-4bfd66c82885 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:09:57.202973 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 631/1241] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:09:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 06:09:57.208729 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bac304-7eca-438c-88c9-952990e3e092 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:09:57.237437 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:09:57.309858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bac304-7eca-438c-88c9-952990e3e092 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:09:57.344843 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e2c41-e674-432b-8be5-225dadaf1bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:09:57.345853 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e2c41-e674-432b-8be5-225dadaf1bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:57.346182 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e2c41-e674-432b-8be5-225dadaf1bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:57.346458 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e2c41-e674-432b-8be5-225dadaf1bd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:57.356714 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e2c41-e674-432b-8be5-225dadaf1bd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:09:57.358179 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e2c41-e674-432b-8be5-225dadaf1bd5 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:09:57.375592 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-f55e2c41-e674-432b-8be5-225dadaf1bd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:09:57.375592 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:09:57.375592 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:09:57.375592 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:09:57.375592 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:09:57.375592 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:09:57.375592 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:09:57.375592 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:09:57.375592 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:09:57.375592 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:09:57.375592 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:09:57.375592 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:09:57.375592 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:09:57.375592 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:09:57.375592 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:09:57.376758 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:09:57.376758 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:09:57.376758 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:09:57.376758 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:09:57.376758 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:09:57.376758 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:09:57.376758 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:09:57.376758 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:09:57.376758 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:09:57.376758 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:09:57.376758 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:09:57.376758 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:09:57.376758 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:09:57.376758 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:09:57.376758 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:09:57.376758 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:09:57.376758 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:09:57.377724 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:09:57.377724 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:09:57.377724 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:09:57.377724 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:09:57.377724 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 611/1242] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:09:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:09:57.382549 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac87dd-8b2d-45d1-bb68-4bfd66c82885 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:09:57.475987 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:09:57.482166 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac87dd-8b2d-45d1-bb68-4bfd66c82885 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:09:57.515364 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb7f9e1-d03b-4362-adc3-a0f95a72a8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:09:57.516030 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7f9e1-d03b-4362-adc3-a0f95a72a8b3 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:57.516310 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7f9e1-d03b-4362-adc3-a0f95a72a8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:57.516618 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7f9e1-d03b-4362-adc3-a0f95a72a8b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:57.517337 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb7f9e1-d03b-4362-adc3-a0f95a72a8b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:09:57.517643 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb7f9e1-d03b-4362-adc3-a0f95a72a8b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:09:57.519063 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb7f9e1-d03b-4362-adc3-a0f95a72a8b3 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:09:57.531298 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 632/1243] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:09:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:09:57.537053 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e2c41-e674-432b-8be5-225dadaf1bd5 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:09:57.615098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:09:57.636939 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e2c41-e674-432b-8be5-225dadaf1bd5 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:09:57.670741 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb285fb7-cae9-4172-8f45-77628c76e6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:09:57.671635 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb285fb7-cae9-4172-8f45-77628c76e6bb None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:57.671923 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb285fb7-cae9-4172-8f45-77628c76e6bb None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:57.672202 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb285fb7-cae9-4172-8f45-77628c76e6bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:57.678838 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb285fb7-cae9-4172-8f45-77628c76e6bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:09:57.680277 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb285fb7-cae9-4172-8f45-77628c76e6bb None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:09:57.694468 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-fb285fb7-cae9-4172-8f45-77628c76e6bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:09:57.694468 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:09:57.694468 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:09:57.694468 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:09:57.694468 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:09:57.694468 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:09:57.694468 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:09:57.694468 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:09:57.694468 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:09:57.694468 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:09:57.694468 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:09:57.694468 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:09:57.694468 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 06:09:57.694468 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 06:09:57.694468 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 06:09:57.697819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 06:09:57.697819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:09:57.697819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:09:57.697819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:09:57.697819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:09:57.697819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:09:57.697819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:09:57.697819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:09:57.697819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:09:57.697819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:09:57.697819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:09:57.697819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:09:57.697819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:09:57.697819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:09:57.697819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:09:57.697819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:09:57.697819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:09:57.699616 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 06:09:57.699616 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 06:09:57.699616 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 06:09:57.699616 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 06:09:57.699616 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 06:09:57.699616 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 06:09:57.699616 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:09:57.699616 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:09:57.699616 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 612/1244] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:09:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:09:57.702038 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb7f9e1-d03b-4362-adc3-a0f95a72a8b3 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:09:57.804867 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb7f9e1-d03b-4362-adc3-a0f95a72a8b3 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:09:57.839557 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bba4a2-26c4-4ec6-ab16-4b437cb8004d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:09:57.840241 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bba4a2-26c4-4ec6-ab16-4b437cb8004d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:57.840587 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bba4a2-26c4-4ec6-ab16-4b437cb8004d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:57.840847 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bba4a2-26c4-4ec6-ab16-4b437cb8004d None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:57.841634 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bba4a2-26c4-4ec6-ab16-4b437cb8004d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:09:57.841946 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bba4a2-26c4-4ec6-ab16-4b437cb8004d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:09:57.843389 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bba4a2-26c4-4ec6-ab16-4b437cb8004d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:09:57.855648 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 633/1245] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 18 06:09:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:09:57.861897 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb285fb7-cae9-4172-8f45-77628c76e6bb None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:09:57.966050 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb285fb7-cae9-4172-8f45-77628c76e6bb None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:09:58.001244 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9d281-6985-40fb-afcf-78ae5a8dd4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:09:58.002046 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9d281-6985-40fb-afcf-78ae5a8dd4ee None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:09:58.002333 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9d281-6985-40fb-afcf-78ae5a8dd4ee None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:09:58.002651 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9d281-6985-40fb-afcf-78ae5a8dd4ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:09:58.003483 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc9d281-6985-40fb-afcf-78ae5a8dd4ee None admin] RBAC: Adding query filter params (role.id=ab10744ff77646eab23a769ab89c74aa, scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a, user.id=a3560534427544cdabfd0ba702a0a165) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:09:58.003795 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc9d281-6985-40fb-afcf-78ae5a8dd4ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:09:58.005382 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc9d281-6985-40fb-afcf-78ae5a8dd4ee None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:09:58.012127 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 613/1246] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:09:57 2022] GET /identity/v3/role_assignments?user.id=a3560534427544cdabfd0ba702a0a165&scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a&role.id=ab10744ff77646eab23a769ab89c74aa => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:10:00.081151 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6169a9-6936-4bf5-97b4-86ce9b9aff03 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:00.081692 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6169a9-6936-4bf5-97b4-86ce9b9aff03 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:00.081692 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6169a9-6936-4bf5-97b4-86ce9b9aff03 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:00.083040 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 634/1247] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:10:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:10:00.088659 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fcb069-7a2a-49b9-82dd-8ab9fe3430f2 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:00.089141 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fcb069-7a2a-49b9-82dd-8ab9fe3430f2 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:00.089252 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fcb069-7a2a-49b9-82dd-8ab9fe3430f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:00.094498 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:10:00.107648 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-f9fcb069-7a2a-49b9-82dd-8ab9fe3430f2 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:10:00.108911 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f9fcb069-7a2a-49b9-82dd-8ab9fe3430f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:10:00.180180 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 614/1248] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:10:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:10:00.210368 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6169a9-6936-4bf5-97b4-86ce9b9aff03 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:00.230347 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:10:00.316799 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6169a9-6936-4bf5-97b4-86ce9b9aff03 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:00.354687 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ff482-7139-4be3-bebc-db5a4d641a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:00.355400 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ff482-7139-4be3-bebc-db5a4d641a10 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:00.355689 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ff482-7139-4be3-bebc-db5a4d641a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:00.355959 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ff482-7139-4be3-bebc-db5a4d641a10 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:00.366636 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88ff482-7139-4be3-bebc-db5a4d641a10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:10:00.374509 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-d88ff482-7139-4be3-bebc-db5a4d641a10 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:10:00.374509 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:10:00.374509 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:10:00.374509 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:10:00.374509 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:10:00.374509 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:10:00.374509 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:10:00.374509 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:10:00.374509 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:10:00.374509 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:10:00.374509 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:10:00.374509 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:10:00.374509 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:10:00.374509 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:10:00.374509 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:10:00.374509 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:10:00.375643 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:10:00.375643 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:10:00.375643 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:10:00.375643 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:10:00.375643 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:10:00.375643 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:10:00.375643 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:10:00.375643 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:10:00.375643 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:10:00.375643 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:10:00.375643 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:10:00.375643 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:10:00.375643 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:10:00.375643 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:10:00.375643 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:10:00.375643 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:10:00.376794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:10:00.376794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:10:00.376794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:10:00.376794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:10:00.376794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:10:00.376794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:10:00.376794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:10:00.376794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:10:00.376794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:10:00.376794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:10:00.376794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:10:00.376794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:10:00.376794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 635/1249] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 18 06:10:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:10:00.382287 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fcb069-7a2a-49b9-82dd-8ab9fe3430f2 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:00.506812 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fcb069-7a2a-49b9-82dd-8ab9fe3430f2 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:00.540091 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:10:00.546900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d60ec-fcd4-4902-8584-456c30eceac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:00.547576 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d60ec-fcd4-4902-8584-456c30eceac1 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:00.547862 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d60ec-fcd4-4902-8584-456c30eceac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:00.548138 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d60ec-fcd4-4902-8584-456c30eceac1 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:00.548928 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0d60ec-fcd4-4902-8584-456c30eceac1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:10:00.549241 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0d60ec-fcd4-4902-8584-456c30eceac1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:00.550610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0d60ec-fcd4-4902-8584-456c30eceac1 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:00.564853 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 615/1250] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:10:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:10:00.568235 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ff482-7139-4be3-bebc-db5a4d641a10 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:00.669457 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:10:00.701934 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ff482-7139-4be3-bebc-db5a4d641a10 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:00.747020 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bef9ac-253c-4f66-885e-f37d50bde5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:00.747827 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bef9ac-253c-4f66-885e-f37d50bde5fe None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:00.748114 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bef9ac-253c-4f66-885e-f37d50bde5fe None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:00.748394 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bef9ac-253c-4f66-885e-f37d50bde5fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:00.760387 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bef9ac-253c-4f66-885e-f37d50bde5fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:00.762685 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bef9ac-253c-4f66-885e-f37d50bde5fe None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:00.784651 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-50bef9ac-253c-4f66-885e-f37d50bde5fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:10:00.784651 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:10:00.784651 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:10:00.784651 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:10:00.784651 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:10:00.784651 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:10:00.784651 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:10:00.784651 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:10:00.784651 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:10:00.784651 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:10:00.784651 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:10:00.784651 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:10:00.784651 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:10:00.784651 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:10:00.784651 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:10:00.785814 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:10:00.785814 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:10:00.785814 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:10:00.785814 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:10:00.785814 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:10:00.785814 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:10:00.785814 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:10:00.785814 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:10:00.785814 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:10:00.785814 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:10:00.785814 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:10:00.785814 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:10:00.785814 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:10:00.785814 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:10:00.785814 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:10:00.785814 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:10:00.785814 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:10:00.791427 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:10:00.791427 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:10:00.791427 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:10:00.791427 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:10:00.791427 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 636/1251] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:10:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:10:00.792988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d60ec-fcd4-4902-8584-456c30eceac1 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:00.900511 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d60ec-fcd4-4902-8584-456c30eceac1 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:00.936255 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed519a8-c654-4699-8225-822caa1e9295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:00.936875 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed519a8-c654-4699-8225-822caa1e9295 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:00.937168 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed519a8-c654-4699-8225-822caa1e9295 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:00.937448 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed519a8-c654-4699-8225-822caa1e9295 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:00.938259 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed519a8-c654-4699-8225-822caa1e9295 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:10:00.938571 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed519a8-c654-4699-8225-822caa1e9295 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:00.939942 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed519a8-c654-4699-8225-822caa1e9295 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:00.952867 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 616/1252] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:10:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:10:00.959439 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bef9ac-253c-4f66-885e-f37d50bde5fe None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:01.062479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bef9ac-253c-4f66-885e-f37d50bde5fe None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:01.087777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:10:01.097036 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a55a256-18e1-4630-b314-a9f5e4671c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:01.097833 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55a256-18e1-4630-b314-a9f5e4671c64 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:01.098118 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55a256-18e1-4630-b314-a9f5e4671c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:01.098393 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55a256-18e1-4630-b314-a9f5e4671c64 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:01.106445 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a55a256-18e1-4630-b314-a9f5e4671c64 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:01.107916 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a55a256-18e1-4630-b314-a9f5e4671c64 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:01.122876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-6a55a256-18e1-4630-b314-a9f5e4671c64 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 06:10:01.122876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:10:01.122876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:10:01.122876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:10:01.122876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:10:01.122876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:10:01.122876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:10:01.122876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:10:01.122876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:10:01.122876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:10:01.122876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:10:01.122876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:10:01.122876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 06:10:01.122876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 06:10:01.122876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 06:10:01.122876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 06:10:01.124122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:10:01.124122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:10:01.124122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 06:10:01.124122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:10:01.124122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 06:10:01.124122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:10:01.124122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:10:01.124122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:10:01.124122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:10:01.124122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:10:01.124122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:10:01.124122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:10:01.124122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:10:01.124122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:10:01.124122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:10:01.124122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:10:01.124122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:10:01.124122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:10:01.125505 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:10:01.125505 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:10:01.125505 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 06:10:01.125505 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 06:10:01.125505 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 06:10:01.125505 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 06:10:01.125505 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 06:10:01.125505 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 06:10:01.125505 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 06:10:01.125505 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:10:01.125505 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 637/1253] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:10:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:10:01.129887 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed519a8-c654-4699-8225-822caa1e9295 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:01.187702 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:10:01.232185 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed519a8-c654-4699-8225-822caa1e9295 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:01.268136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b5a6d7-42dd-49ba-8405-879a3c2d5e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:01.268841 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5a6d7-42dd-49ba-8405-879a3c2d5e14 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:01.269159 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5a6d7-42dd-49ba-8405-879a3c2d5e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:01.269448 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5a6d7-42dd-49ba-8405-879a3c2d5e14 None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:01.270287 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b5a6d7-42dd-49ba-8405-879a3c2d5e14 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:10:01.270597 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b5a6d7-42dd-49ba-8405-879a3c2d5e14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:01.272076 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b5a6d7-42dd-49ba-8405-879a3c2d5e14 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:01.290674 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 617/1254] 10.222.0.40 () {56 vars in 1143 bytes} [Mon Jul 18 06:10:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:10:01.297017 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a55a256-18e1-4630-b314-a9f5e4671c64 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:01.398822 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a55a256-18e1-4630-b314-a9f5e4671c64 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:01.434171 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3de18-ce78-463c-aa70-d2222da453ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:01.434821 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3de18-ce78-463c-aa70-d2222da453ff None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:01.435105 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3de18-ce78-463c-aa70-d2222da453ff None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:01.435382 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3de18-ce78-463c-aa70-d2222da453ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:01.491361 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:10:01.503240 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b3de18-ce78-463c-aa70-d2222da453ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:01.504927 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b3de18-ce78-463c-aa70-d2222da453ff None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:01.608933 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 638/1255] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:10:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 06:10:01.614885 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b5a6d7-42dd-49ba-8405-879a3c2d5e14 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:01.716086 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b5a6d7-42dd-49ba-8405-879a3c2d5e14 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:01.729147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:10:01.751456 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ffb6d-2e0c-4e62-8857-e2f59cdd7813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:01.752175 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ffb6d-2e0c-4e62-8857-e2f59cdd7813 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:01.752487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ffb6d-2e0c-4e62-8857-e2f59cdd7813 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:01.752770 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ffb6d-2e0c-4e62-8857-e2f59cdd7813 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:01.763465 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9ffb6d-2e0c-4e62-8857-e2f59cdd7813 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:01.764945 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9ffb6d-2e0c-4e62-8857-e2f59cdd7813 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:01.782900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-dc9ffb6d-2e0c-4e62-8857-e2f59cdd7813 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:10:01.782900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:10:01.782900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:10:01.782900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:10:01.782900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:10:01.782900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:10:01.782900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:10:01.782900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:10:01.782900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:10:01.782900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:10:01.782900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:10:01.782900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:10:01.782900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:10:01.782900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:10:01.782900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:10:01.786106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:10:01.786106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:10:01.786106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:10:01.786106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:10:01.786106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:10:01.786106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:10:01.786106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:10:01.786106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:10:01.786106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:10:01.786106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:10:01.786106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:10:01.786106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:10:01.786106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:10:01.786106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:10:01.786106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:10:01.786106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:10:01.786106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:10:01.787399 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:10:01.787399 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:10:01.787399 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:10:01.787399 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:10:01.787399 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 618/1256] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:10:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:10:01.790036 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3de18-ce78-463c-aa70-d2222da453ff None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:01.876713 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:10:01.888958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3de18-ce78-463c-aa70-d2222da453ff None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:01.922948 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae28f570-db88-45c3-83b5-6660fea995c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:01.923615 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae28f570-db88-45c3-83b5-6660fea995c6 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:01.923906 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae28f570-db88-45c3-83b5-6660fea995c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:01.924185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae28f570-db88-45c3-83b5-6660fea995c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:01.924952 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae28f570-db88-45c3-83b5-6660fea995c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:10:01.925259 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae28f570-db88-45c3-83b5-6660fea995c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:01.926640 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae28f570-db88-45c3-83b5-6660fea995c6 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:01.938114 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 639/1257] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:10:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:10:01.943775 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ffb6d-2e0c-4e62-8857-e2f59cdd7813 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:02.043158 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ffb6d-2e0c-4e62-8857-e2f59cdd7813 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:02.076684 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645dce4-2ab8-437e-8e89-b8cece0ce4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:02.077468 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645dce4-2ab8-437e-8e89-b8cece0ce4a3 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:02.077757 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645dce4-2ab8-437e-8e89-b8cece0ce4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:02.078037 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645dce4-2ab8-437e-8e89-b8cece0ce4a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:02.084851 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2645dce4-2ab8-437e-8e89-b8cece0ce4a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:02.086326 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2645dce4-2ab8-437e-8e89-b8cece0ce4a3 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:02.100942 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-2645dce4-2ab8-437e-8e89-b8cece0ce4a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:10:02.100942 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:10:02.100942 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:10:02.100942 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:10:02.100942 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:10:02.100942 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:10:02.100942 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:10:02.100942 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:10:02.100942 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:10:02.100942 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:10:02.100942 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:10:02.100942 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:10:02.100942 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 06:10:02.100942 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 06:10:02.100942 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 06:10:02.102028 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 06:10:02.102028 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:10:02.102028 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:10:02.102028 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:10:02.102028 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:10:02.102028 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:10:02.102028 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:10:02.102028 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:10:02.102028 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:10:02.102028 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:10:02.102028 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:10:02.102028 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:10:02.102028 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:10:02.102028 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:10:02.102028 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:10:02.102028 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:10:02.102028 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:10:02.103150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 06:10:02.103150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 06:10:02.103150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 06:10:02.103150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 06:10:02.103150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 06:10:02.103150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 06:10:02.103150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:10:02.103150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:10:02.103150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 619/1258] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:10:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:10:02.108658 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae28f570-db88-45c3-83b5-6660fea995c6 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:02.210773 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae28f570-db88-45c3-83b5-6660fea995c6 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:02.243755 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a318ea-e01a-426a-9fe6-cec265b36303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:02.244432 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a318ea-e01a-426a-9fe6-cec265b36303 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:02.244765 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a318ea-e01a-426a-9fe6-cec265b36303 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:02.245566 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a318ea-e01a-426a-9fe6-cec265b36303 None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:02.246345 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a318ea-e01a-426a-9fe6-cec265b36303 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:10:02.246662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a318ea-e01a-426a-9fe6-cec265b36303 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:02.248098 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a318ea-e01a-426a-9fe6-cec265b36303 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:02.260220 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 640/1259] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 18 06:10:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:10:02.266936 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645dce4-2ab8-437e-8e89-b8cece0ce4a3 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:02.302013 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:10:02.366049 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645dce4-2ab8-437e-8e89-b8cece0ce4a3 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:02.399708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd6c430-a71c-42be-9431-8f1d05a43782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:02.400360 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd6c430-a71c-42be-9431-8f1d05a43782 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:02.400676 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd6c430-a71c-42be-9431-8f1d05a43782 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:02.400960 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd6c430-a71c-42be-9431-8f1d05a43782 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:02.401956 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd6c430-a71c-42be-9431-8f1d05a43782 None admin] RBAC: Adding query filter params (role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc, scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a, user.id=a3560534427544cdabfd0ba702a0a165) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:10:02.402272 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd6c430-a71c-42be-9431-8f1d05a43782 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:02.403700 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd6c430-a71c-42be-9431-8f1d05a43782 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:02.409448 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 620/1260] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:10:02 2022] GET /identity/v3/role_assignments?user.id=a3560534427544cdabfd0ba702a0a165&scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a&role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:10:04.439825 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90209f1a-a8ab-4801-881b-f0fdbd242b79 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:04.440382 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90209f1a-a8ab-4801-881b-f0fdbd242b79 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:04.440382 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90209f1a-a8ab-4801-881b-f0fdbd242b79 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:04.441801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 641/1261] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:10:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:10:04.447029 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25955d6e-bc6a-40e8-9f24-e13d7a2ce78d None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:04.447449 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25955d6e-bc6a-40e8-9f24-e13d7a2ce78d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:04.447602 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25955d6e-bc6a-40e8-9f24-e13d7a2ce78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:04.465715 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-25955d6e-bc6a-40e8-9f24-e13d7a2ce78d None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:10:04.466942 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-25955d6e-bc6a-40e8-9f24-e13d7a2ce78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:10:04.535493 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 621/1262] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:10:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:10:04.558551 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90209f1a-a8ab-4801-881b-f0fdbd242b79 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:04.569576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:10:04.661900 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90209f1a-a8ab-4801-881b-f0fdbd242b79 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:04.705364 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4630cc-18fe-4059-97c3-889d3b7210a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:04.706087 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4630cc-18fe-4059-97c3-889d3b7210a3 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:04.706371 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4630cc-18fe-4059-97c3-889d3b7210a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:04.706673 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4630cc-18fe-4059-97c3-889d3b7210a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:04.716757 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4630cc-18fe-4059-97c3-889d3b7210a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:10:04.724266 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-1e4630cc-18fe-4059-97c3-889d3b7210a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:10:04.724266 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:10:04.724266 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:10:04.724266 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:10:04.724266 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:10:04.724266 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:10:04.724266 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:10:04.724266 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:10:04.724266 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:10:04.724266 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:10:04.724266 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:10:04.724266 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:10:04.724266 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:10:04.724266 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:10:04.724266 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:10:04.724266 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:10:04.725447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:10:04.725447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:10:04.725447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:10:04.725447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:10:04.725447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:10:04.725447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:10:04.725447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:10:04.725447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:10:04.725447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:10:04.725447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:10:04.725447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:10:04.725447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:10:04.725447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:10:04.725447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:10:04.725447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:10:04.725447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:10:04.726605 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:10:04.726605 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:10:04.726605 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:10:04.726605 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:10:04.726605 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:10:04.726605 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:10:04.726605 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:10:04.726605 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:10:04.726605 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:10:04.726605 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:10:04.726605 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:10:04.726605 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:10:04.726605 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 642/1263] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 18 06:10:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:10:04.731635 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25955d6e-bc6a-40e8-9f24-e13d7a2ce78d None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:04.751203 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:10:04.832438 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25955d6e-bc6a-40e8-9f24-e13d7a2ce78d None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:04.868051 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8879bf8d-6a19-45fe-9218-b94c36344b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:04.868750 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8879bf8d-6a19-45fe-9218-b94c36344b08 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:04.869027 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8879bf8d-6a19-45fe-9218-b94c36344b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:04.869300 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8879bf8d-6a19-45fe-9218-b94c36344b08 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:04.870791 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8879bf8d-6a19-45fe-9218-b94c36344b08 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:10:04.871100 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8879bf8d-6a19-45fe-9218-b94c36344b08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:04.872483 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8879bf8d-6a19-45fe-9218-b94c36344b08 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:04.880347 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 622/1264] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:10:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:10:04.886752 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4630cc-18fe-4059-97c3-889d3b7210a3 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:04.957473 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:10:04.991011 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4630cc-18fe-4059-97c3-889d3b7210a3 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:05.024545 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f09bc19-d998-4420-b526-a87540edcbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:05.025341 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09bc19-d998-4420-b526-a87540edcbfa None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:05.025631 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09bc19-d998-4420-b526-a87540edcbfa None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:05.025956 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09bc19-d998-4420-b526-a87540edcbfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:05.037066 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f09bc19-d998-4420-b526-a87540edcbfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:05.038550 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f09bc19-d998-4420-b526-a87540edcbfa None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:05.056158 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-3f09bc19-d998-4420-b526-a87540edcbfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:10:05.056158 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:10:05.056158 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:10:05.056158 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:10:05.056158 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:10:05.056158 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:10:05.056158 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:10:05.056158 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:10:05.056158 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:10:05.056158 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:10:05.056158 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:10:05.056158 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:10:05.056158 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:10:05.056158 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:10:05.056158 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:10:05.059027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:10:05.059027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:10:05.059027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:10:05.059027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:10:05.059027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:10:05.059027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:10:05.059027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:10:05.059027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:10:05.059027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:10:05.059027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:10:05.059027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:10:05.059027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:10:05.059027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:10:05.059027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:10:05.059027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:10:05.059027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:10:05.059027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:10:05.060079 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:10:05.060079 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:10:05.060079 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:10:05.060079 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:10:05.060079 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 643/1265] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:10:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:10:05.064178 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8879bf8d-6a19-45fe-9218-b94c36344b08 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:05.146993 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:10:05.166062 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8879bf8d-6a19-45fe-9218-b94c36344b08 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:05.199249 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f660d21e-4f7e-49b5-814e-4559069cf774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:05.199907 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660d21e-4f7e-49b5-814e-4559069cf774 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:05.200198 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660d21e-4f7e-49b5-814e-4559069cf774 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:05.200658 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660d21e-4f7e-49b5-814e-4559069cf774 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:05.201392 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f660d21e-4f7e-49b5-814e-4559069cf774 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:10:05.201705 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f660d21e-4f7e-49b5-814e-4559069cf774 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:05.203070 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f660d21e-4f7e-49b5-814e-4559069cf774 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:05.215374 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 623/1266] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:10:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:10:05.221052 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f09bc19-d998-4420-b526-a87540edcbfa None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:05.324226 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f09bc19-d998-4420-b526-a87540edcbfa None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:05.345150 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:10:05.359595 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5091c1-b76b-4dd3-9221-59ca2139b894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:05.360405 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5091c1-b76b-4dd3-9221-59ca2139b894 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:05.360732 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5091c1-b76b-4dd3-9221-59ca2139b894 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:05.361019 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5091c1-b76b-4dd3-9221-59ca2139b894 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:05.368398 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5091c1-b76b-4dd3-9221-59ca2139b894 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:05.370000 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5091c1-b76b-4dd3-9221-59ca2139b894 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:05.384284 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-8e5091c1-b76b-4dd3-9221-59ca2139b894 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 06:10:05.384284 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:10:05.384284 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:10:05.384284 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:10:05.384284 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:10:05.384284 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:10:05.384284 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:10:05.384284 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:10:05.384284 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:10:05.384284 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:10:05.384284 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:10:05.384284 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:10:05.384284 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 06:10:05.384284 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 06:10:05.384284 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 06:10:05.384284 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 06:10:05.385504 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:10:05.385504 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:10:05.385504 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 06:10:05.385504 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:10:05.385504 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 06:10:05.385504 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:10:05.385504 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:10:05.385504 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:10:05.385504 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:10:05.385504 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:10:05.385504 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:10:05.385504 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:10:05.385504 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:10:05.385504 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:10:05.385504 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:10:05.385504 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:10:05.385504 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:10:05.385504 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:10:05.386481 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:10:05.386481 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:10:05.386481 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 06:10:05.386481 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 06:10:05.386481 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 06:10:05.386481 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 06:10:05.386481 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 06:10:05.386481 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 06:10:05.386481 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 06:10:05.386481 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:10:05.386481 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 644/1267] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:10:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:10:05.391769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f660d21e-4f7e-49b5-814e-4559069cf774 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:05.491258 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f660d21e-4f7e-49b5-814e-4559069cf774 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:05.524597 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0753dea-218b-49d7-8e37-76f10285a494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:05.525334 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0753dea-218b-49d7-8e37-76f10285a494 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:05.525610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0753dea-218b-49d7-8e37-76f10285a494 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:05.525885 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0753dea-218b-49d7-8e37-76f10285a494 None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:05.527406 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0753dea-218b-49d7-8e37-76f10285a494 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:10:05.527720 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0753dea-218b-49d7-8e37-76f10285a494 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:05.529215 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0753dea-218b-49d7-8e37-76f10285a494 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:05.538069 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:10:05.546783 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 624/1268] 10.222.0.40 () {56 vars in 1143 bytes} [Mon Jul 18 06:10:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:10:05.552702 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5091c1-b76b-4dd3-9221-59ca2139b894 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:05.653517 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5091c1-b76b-4dd3-9221-59ca2139b894 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:05.688083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeff62f-1c1d-4d39-ae28-2353e92af5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:05.688926 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeff62f-1c1d-4d39-ae28-2353e92af5e8 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:05.689213 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeff62f-1c1d-4d39-ae28-2353e92af5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:05.689502 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeff62f-1c1d-4d39-ae28-2353e92af5e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:05.700710 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeff62f-1c1d-4d39-ae28-2353e92af5e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:05.702162 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeff62f-1c1d-4d39-ae28-2353e92af5e8 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:05.719836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-caeff62f-1c1d-4d39-ae28-2353e92af5e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:10:05.719836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:10:05.719836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:10:05.719836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:10:05.719836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:10:05.719836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:10:05.719836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:10:05.719836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:10:05.719836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:10:05.719836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:10:05.719836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:10:05.719836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:10:05.719836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:10:05.719836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:10:05.719836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:10:05.720946 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:10:05.720946 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:10:05.720946 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:10:05.720946 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:10:05.720946 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:10:05.720946 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:10:05.720946 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:10:05.720946 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:10:05.720946 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:10:05.720946 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:10:05.720946 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:10:05.720946 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:10:05.720946 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:10:05.720946 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:10:05.720946 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:10:05.720946 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:10:05.720946 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:10:05.722055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:10:05.722055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:10:05.722055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:10:05.722055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:10:05.722055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 645/1269] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:10:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:10:05.727152 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0753dea-218b-49d7-8e37-76f10285a494 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:05.828996 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0753dea-218b-49d7-8e37-76f10285a494 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:05.862733 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611e36d-17d0-496a-93b1-520467051742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:05.863436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611e36d-17d0-496a-93b1-520467051742 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:05.863703 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611e36d-17d0-496a-93b1-520467051742 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:05.863985 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611e36d-17d0-496a-93b1-520467051742 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:05.864769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611e36d-17d0-496a-93b1-520467051742 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:10:05.865137 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611e36d-17d0-496a-93b1-520467051742 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:05.866524 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611e36d-17d0-496a-93b1-520467051742 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:05.878347 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 625/1270] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:10:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:10:05.884344 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeff62f-1c1d-4d39-ae28-2353e92af5e8 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:05.916433 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:10:05.984820 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeff62f-1c1d-4d39-ae28-2353e92af5e8 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:06.018095 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2050ad03-4a88-463f-80db-60b78b5e2af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:06.021625 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050ad03-4a88-463f-80db-60b78b5e2af1 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:06.021625 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050ad03-4a88-463f-80db-60b78b5e2af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:06.021625 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050ad03-4a88-463f-80db-60b78b5e2af1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:06.026162 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2050ad03-4a88-463f-80db-60b78b5e2af1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:06.027605 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2050ad03-4a88-463f-80db-60b78b5e2af1 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:06.041992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-2050ad03-4a88-463f-80db-60b78b5e2af1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:10:06.041992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:10:06.041992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:10:06.041992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:10:06.041992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:10:06.041992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:10:06.041992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:10:06.041992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:10:06.041992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:10:06.041992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:10:06.041992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:10:06.041992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:10:06.041992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 06:10:06.041992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 06:10:06.041992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 06:10:06.043130 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 06:10:06.043130 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:10:06.043130 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:10:06.043130 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:10:06.043130 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:10:06.043130 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:10:06.043130 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:10:06.043130 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:10:06.043130 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:10:06.043130 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:10:06.043130 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:10:06.043130 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:10:06.043130 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:10:06.043130 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:10:06.043130 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:10:06.043130 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:10:06.043130 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:10:06.044391 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 06:10:06.044391 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 06:10:06.044391 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 06:10:06.044391 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 06:10:06.044391 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 06:10:06.044391 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 06:10:06.044391 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:10:06.044391 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:10:06.044391 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 646/1271] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:10:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:10:06.049271 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611e36d-17d0-496a-93b1-520467051742 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:06.107886 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:10:06.149150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611e36d-17d0-496a-93b1-520467051742 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:06.184882 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b88fa-31d3-4e29-ad7d-a497b62ef733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:06.185538 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b88fa-31d3-4e29-ad7d-a497b62ef733 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:06.185828 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b88fa-31d3-4e29-ad7d-a497b62ef733 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:06.186104 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b88fa-31d3-4e29-ad7d-a497b62ef733 None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:06.186895 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7b88fa-31d3-4e29-ad7d-a497b62ef733 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:10:06.187204 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7b88fa-31d3-4e29-ad7d-a497b62ef733 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:06.188668 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7b88fa-31d3-4e29-ad7d-a497b62ef733 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:06.201569 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 626/1272] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 18 06:10:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:10:06.207728 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2050ad03-4a88-463f-80db-60b78b5e2af1 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:06.303859 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:10:06.313006 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2050ad03-4a88-463f-80db-60b78b5e2af1 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:06.346295 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a8322-2ba7-4ad0-9a60-7d8522b5a7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:06.347189 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a8322-2ba7-4ad0-9a60-7d8522b5a7da None admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:06.347481 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a8322-2ba7-4ad0-9a60-7d8522b5a7da None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:06.347762 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a8322-2ba7-4ad0-9a60-7d8522b5a7da None admin] PATH_INFO: `/v3/projects/8ed50a8a3e184bde9269d7b0d9573d4a/users/a3560534427544cdabfd0ba702a0a165/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:06.349759 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a8322-2ba7-4ad0-9a60-7d8522b5a7da None admin] RBAC: Authorizing `identity:create_grant(project_id=8ed50a8a3e184bde9269d7b0d9573d4a, user_id=a3560534427544cdabfd0ba702a0a165, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:06.351233 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a8322-2ba7-4ad0-9a60-7d8522b5a7da None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:06.456004 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 647/1273] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 18 06:10:06 2022] PUT /identity/v3/projects/8ed50a8a3e184bde9269d7b0d9573d4a/users/a3560534427544cdabfd0ba702a0a165/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:10:08.469701 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f2222-8695-4689-a369-7b581d6851a8 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:08.470261 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f2222-8695-4689-a369-7b581d6851a8 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:08.470261 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f2222-8695-4689-a369-7b581d6851a8 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:08.471635 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 627/1274] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:10:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:10:08.477252 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b5b8e-0c16-434d-9a4a-9237d7e66bdd None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:08.477774 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b5b8e-0c16-434d-9a4a-9237d7e66bdd None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:08.477903 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b5b8e-0c16-434d-9a4a-9237d7e66bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:08.494319 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-933b5b8e-0c16-434d-9a4a-9237d7e66bdd None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:10:08.495575 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-933b5b8e-0c16-434d-9a4a-9237d7e66bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:10:08.563640 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 648/1275] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:10:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:10:08.587784 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f2222-8695-4689-a369-7b581d6851a8 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:08.691113 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f2222-8695-4689-a369-7b581d6851a8 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:08.709323 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:10:08.726502 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf137cc-28b3-456a-85fe-e7e7b27f2a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:08.727234 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf137cc-28b3-456a-85fe-e7e7b27f2a4d None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:08.727515 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf137cc-28b3-456a-85fe-e7e7b27f2a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:08.727788 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf137cc-28b3-456a-85fe-e7e7b27f2a4d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:08.739164 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf137cc-28b3-456a-85fe-e7e7b27f2a4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 06:10:08.747916 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-daf137cc-28b3-456a-85fe-e7e7b27f2a4d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:10:08.747916 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:10:08.747916 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:10:08.747916 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:10:08.747916 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:10:08.747916 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:10:08.747916 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:10:08.747916 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:10:08.747916 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:10:08.747916 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:10:08.747916 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:10:08.747916 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:10:08.747916 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 06:10:08.747916 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 06:10:08.747916 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 06:10:08.747916 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 06:10:08.749085 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 06:10:08.749085 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 06:10:08.749085 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 06:10:08.749085 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 06:10:08.749085 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:10:08.749085 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:10:08.749085 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:10:08.749085 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:10:08.749085 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:10:08.749085 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:10:08.749085 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:10:08.749085 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:10:08.749085 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:10:08.749085 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:10:08.749085 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:10:08.749085 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:10:08.750005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:10:08.750005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:10:08.750005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:10:08.750005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:10:08.750005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 06:10:08.750005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 06:10:08.750005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 06:10:08.750005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 06:10:08.750005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 06:10:08.750005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 06:10:08.750005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 06:10:08.750005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:10:08.750005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 628/1276] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 18 06:10:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:10:08.755357 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b5b8e-0c16-434d-9a4a-9237d7e66bdd None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:08.838804 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:10:08.857354 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b5b8e-0c16-434d-9a4a-9237d7e66bdd None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:08.889891 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576664a6-cb95-4440-b2cf-64fe220db6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:08.890660 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576664a6-cb95-4440-b2cf-64fe220db6e2 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:08.890836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576664a6-cb95-4440-b2cf-64fe220db6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:08.891164 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576664a6-cb95-4440-b2cf-64fe220db6e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:08.891913 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576664a6-cb95-4440-b2cf-64fe220db6e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:10:08.892222 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576664a6-cb95-4440-b2cf-64fe220db6e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:08.893638 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576664a6-cb95-4440-b2cf-64fe220db6e2 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:08.901612 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 649/1277] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 18 06:10:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:10:08.908236 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf137cc-28b3-456a-85fe-e7e7b27f2a4d None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:09.009888 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf137cc-28b3-456a-85fe-e7e7b27f2a4d None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:09.043674 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a0eb8-9a5c-4308-8e27-44f2b8bec673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:09.045354 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a0eb8-9a5c-4308-8e27-44f2b8bec673 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:09.045354 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a0eb8-9a5c-4308-8e27-44f2b8bec673 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:09.045354 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a0eb8-9a5c-4308-8e27-44f2b8bec673 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:09.055953 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5a0eb8-9a5c-4308-8e27-44f2b8bec673 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:09.057417 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5a0eb8-9a5c-4308-8e27-44f2b8bec673 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:09.075005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-6c5a0eb8-9a5c-4308-8e27-44f2b8bec673 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:10:09.075005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:10:09.075005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:10:09.075005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:10:09.075005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:10:09.075005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:10:09.075005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:10:09.075005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:10:09.075005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:10:09.075005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:10:09.075005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:10:09.075005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:10:09.075005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:10:09.075005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:10:09.075005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:10:09.076312 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:10:09.076312 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:10:09.076312 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:10:09.076312 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:10:09.076312 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:10:09.076312 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:10:09.076312 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:10:09.076312 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:10:09.076312 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:10:09.076312 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:10:09.076312 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:10:09.076312 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:10:09.076312 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:10:09.076312 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:10:09.076312 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:10:09.076312 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:10:09.076312 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:10:09.086489 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:10:09.086489 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:10:09.086489 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:10:09.086489 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:10:09.086489 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 629/1278] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:10:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:10:09.086786 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576664a6-cb95-4440-b2cf-64fe220db6e2 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:09.185830 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576664a6-cb95-4440-b2cf-64fe220db6e2 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:09.219906 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd572ef-2c32-418b-a596-b299a85ce946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:09.220597 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd572ef-2c32-418b-a596-b299a85ce946 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:09.220885 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd572ef-2c32-418b-a596-b299a85ce946 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:09.221164 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd572ef-2c32-418b-a596-b299a85ce946 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:09.221928 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd572ef-2c32-418b-a596-b299a85ce946 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:10:09.222239 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd572ef-2c32-418b-a596-b299a85ce946 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:09.223658 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd572ef-2c32-418b-a596-b299a85ce946 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:09.235338 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 650/1279] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:10:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:10:09.241221 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a0eb8-9a5c-4308-8e27-44f2b8bec673 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:09.272573 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:10:09.348280 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a0eb8-9a5c-4308-8e27-44f2b8bec673 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:09.380120 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9391c0-2a84-4081-a12a-fd5b2fa54bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:09.381002 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9391c0-2a84-4081-a12a-fd5b2fa54bac None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:09.381290 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9391c0-2a84-4081-a12a-fd5b2fa54bac None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:09.381571 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9391c0-2a84-4081-a12a-fd5b2fa54bac None admin] PATH_INFO: `/v3/users/placement` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:09.389372 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9391c0-2a84-4081-a12a-fd5b2fa54bac None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:09.390792 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9391c0-2a84-4081-a12a-fd5b2fa54bac None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:09.406251 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-3d9391c0-2a84-4081-a12a-fd5b2fa54bac None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 06:10:09.406251 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:10:09.406251 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:10:09.406251 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:10:09.406251 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:10:09.406251 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:10:09.406251 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:10:09.406251 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:10:09.406251 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:10:09.406251 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:10:09.406251 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:10:09.406251 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:10:09.406251 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 06:10:09.406251 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 06:10:09.406251 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 06:10:09.406251 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 06:10:09.407385 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:10:09.407385 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:10:09.407385 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 06:10:09.407385 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:10:09.407385 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 06:10:09.407385 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 06:10:09.407385 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:10:09.407385 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:10:09.407385 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:10:09.407385 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:10:09.407385 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:10:09.407385 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:10:09.407385 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:10:09.407385 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:10:09.407385 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:10:09.407385 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:10:09.407385 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:10:09.407385 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:10:09.408614 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:10:09.408614 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:10:09.408614 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 06:10:09.408614 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 06:10:09.408614 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 06:10:09.408614 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 06:10:09.408614 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 06:10:09.408614 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 06:10:09.408614 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 06:10:09.408614 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:10:09.408614 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 630/1280] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:10:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:10:09.414136 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd572ef-2c32-418b-a596-b299a85ce946 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:09.418107 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:10:09.514131 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd572ef-2c32-418b-a596-b299a85ce946 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:09.550822 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c742bf6-9b52-4615-bb30-d140143ed9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:09.551424 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c742bf6-9b52-4615-bb30-d140143ed9b0 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:09.551712 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c742bf6-9b52-4615-bb30-d140143ed9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:09.551998 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c742bf6-9b52-4615-bb30-d140143ed9b0 None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:09.552979 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c742bf6-9b52-4615-bb30-d140143ed9b0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:10:09.553295 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c742bf6-9b52-4615-bb30-d140143ed9b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:09.554740 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c742bf6-9b52-4615-bb30-d140143ed9b0 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:09.572178 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 651/1281] 10.222.0.40 () {56 vars in 1143 bytes} [Mon Jul 18 06:10:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:10:09.578615 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9391c0-2a84-4081-a12a-fd5b2fa54bac None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:09.672896 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:10:09.682824 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9391c0-2a84-4081-a12a-fd5b2fa54bac None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:09.717004 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6821eab7-8372-40a0-aa52-e3cf6d74a5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:09.717645 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821eab7-8372-40a0-aa52-e3cf6d74a5f7 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:09.717940 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821eab7-8372-40a0-aa52-e3cf6d74a5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:09.718243 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821eab7-8372-40a0-aa52-e3cf6d74a5f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:09.787051 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6821eab7-8372-40a0-aa52-e3cf6d74a5f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:09.788577 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6821eab7-8372-40a0-aa52-e3cf6d74a5f7 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:09.881928 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:10:09.892765 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 631/1282] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 18 06:10:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 06:10:09.898665 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c742bf6-9b52-4615-bb30-d140143ed9b0 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:09.961395 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:10:10.000941 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c742bf6-9b52-4615-bb30-d140143ed9b0 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:10.033598 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea7889f-1887-4419-8aae-551d300a66be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:10.034379 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7889f-1887-4419-8aae-551d300a66be None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:10.034664 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7889f-1887-4419-8aae-551d300a66be None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:10.034942 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7889f-1887-4419-8aae-551d300a66be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:10.045353 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea7889f-1887-4419-8aae-551d300a66be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:10.046850 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea7889f-1887-4419-8aae-551d300a66be None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:10.064482 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-7ea7889f-1887-4419-8aae-551d300a66be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:10:10.064482 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:10:10.064482 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:10:10.064482 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:10:10.064482 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:10:10.064482 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:10:10.064482 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:10:10.064482 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:10:10.064482 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:10:10.064482 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:10:10.064482 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:10:10.064482 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:10:10.064482 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 06:10:10.064482 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 06:10:10.064482 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 06:10:10.073535 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 06:10:10.073535 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:10:10.073535 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:10:10.073535 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:10:10.073535 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:10:10.073535 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:10:10.073535 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:10:10.073535 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:10:10.073535 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:10:10.073535 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:10:10.073535 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:10:10.073535 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:10:10.073535 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:10:10.073535 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:10:10.073535 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:10:10.073535 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:10:10.073535 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:10:10.074718 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 06:10:10.074718 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 06:10:10.074718 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 06:10:10.074718 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:10:10.074718 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 652/1283] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:10:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:10:10.075063 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6821eab7-8372-40a0-aa52-e3cf6d74a5f7 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:10.171836 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6821eab7-8372-40a0-aa52-e3cf6d74a5f7 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:10.206080 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c489d1-d93d-4e67-a284-6183610245be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:10.206731 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c489d1-d93d-4e67-a284-6183610245be None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:10.207020 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c489d1-d93d-4e67-a284-6183610245be None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:10.207301 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c489d1-d93d-4e67-a284-6183610245be None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:10.208021 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c489d1-d93d-4e67-a284-6183610245be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:10:10.208332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c489d1-d93d-4e67-a284-6183610245be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:10.209908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c489d1-d93d-4e67-a284-6183610245be None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:10.226031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 632/1284] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 18 06:10:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:10:10.227892 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea7889f-1887-4419-8aae-551d300a66be None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:10.332720 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea7889f-1887-4419-8aae-551d300a66be None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:10.346813 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:10:10.367364 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217445a1-d432-4697-8e67-d95f80e473cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:10.368165 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217445a1-d432-4697-8e67-d95f80e473cf None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:10.368488 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217445a1-d432-4697-8e67-d95f80e473cf None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:10.368767 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217445a1-d432-4697-8e67-d95f80e473cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:10.376241 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217445a1-d432-4697-8e67-d95f80e473cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:10.377912 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217445a1-d432-4697-8e67-d95f80e473cf None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:10.392617 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-217445a1-d432-4697-8e67-d95f80e473cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:10:10.392617 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:10:10.392617 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:10:10.392617 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:10:10.392617 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:10:10.392617 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:10:10.392617 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:10:10.392617 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:10:10.392617 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:10:10.392617 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:10:10.392617 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:10:10.392617 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:10:10.392617 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 06:10:10.392617 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 06:10:10.392617 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 06:10:10.393692 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 06:10:10.393692 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:10:10.393692 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:10:10.393692 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:10:10.393692 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:10:10.393692 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:10:10.393692 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:10:10.393692 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:10:10.393692 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 18 06:10:10.393692 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:10:10.393692 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:10:10.393692 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:10:10.393692 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:10:10.393692 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:10:10.393692 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:10:10.393692 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:10:10.393692 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:10:10.394659 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 06:10:10.394659 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 06:10:10.394659 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 06:10:10.394659 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 06:10:10.394659 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 06:10:10.394659 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 06:10:10.394659 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 06:10:10.394659 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 18 06:10:10.394659 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 653/1285] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 18 06:10:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:10:10.399770 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c489d1-d93d-4e67-a284-6183610245be None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:10.450218 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:10:10.499051 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c489d1-d93d-4e67-a284-6183610245be None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:10.533374 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23394dbd-489d-498c-85fb-9b0d7afd51ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:10.534037 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23394dbd-489d-498c-85fb-9b0d7afd51ae None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:10.536449 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23394dbd-489d-498c-85fb-9b0d7afd51ae None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:10.536449 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23394dbd-489d-498c-85fb-9b0d7afd51ae None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:10.536449 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23394dbd-489d-498c-85fb-9b0d7afd51ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:10:10.536449 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23394dbd-489d-498c-85fb-9b0d7afd51ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:10.537274 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23394dbd-489d-498c-85fb-9b0d7afd51ae None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:10.550334 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 633/1286] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 18 06:10:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:10:10.556683 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217445a1-d432-4697-8e67-d95f80e473cf None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:10.655746 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217445a1-d432-4697-8e67-d95f80e473cf None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:10.693997 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b074e-e6b5-4d48-a471-8dbbd7aecca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:10.694657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b074e-e6b5-4d48-a471-8dbbd7aecca4 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:10.694942 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b074e-e6b5-4d48-a471-8dbbd7aecca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:10.695220 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b074e-e6b5-4d48-a471-8dbbd7aecca4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:10.696048 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b074e-e6b5-4d48-a471-8dbbd7aecca4 None admin] RBAC: Adding query filter params (role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc, scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a, user.id=a3560534427544cdabfd0ba702a0a165) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:10:10.696362 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b074e-e6b5-4d48-a471-8dbbd7aecca4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:10.697810 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b074e-e6b5-4d48-a471-8dbbd7aecca4 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:10.704482 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 654/1287] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 18 06:10:10 2022] GET /identity/v3/role_assignments?user.id=a3560534427544cdabfd0ba702a0a165&scope.project.id=8ed50a8a3e184bde9269d7b0d9573d4a&role.id=13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:10:12.775018 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1519b9f5-8fa7-49fb-af0a-68ba312bc2f5 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:12.775569 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1519b9f5-8fa7-49fb-af0a-68ba312bc2f5 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:12.775569 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1519b9f5-8fa7-49fb-af0a-68ba312bc2f5 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:12.776981 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 634/1288] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:10:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:10:12.782748 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb099d6-a16d-47da-aeb4-224d8af62a33 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:12.783159 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb099d6-a16d-47da-aeb4-224d8af62a33 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:12.783286 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb099d6-a16d-47da-aeb4-224d8af62a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:12.800066 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-7bb099d6-a16d-47da-aeb4-224d8af62a33 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:10:12.801277 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-7bb099d6-a16d-47da-aeb4-224d8af62a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:10:12.831184 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:10:12.867130 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 655/1289] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:10:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:10:12.890844 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1519b9f5-8fa7-49fb-af0a-68ba312bc2f5 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:12.994439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1519b9f5-8fa7-49fb-af0a-68ba312bc2f5 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:13.005660 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:10:13.028297 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff4793-3641-4fd9-9688-f39f243dd205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:13.029168 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff4793-3641-4fd9-9688-f39f243dd205 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:13.029456 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff4793-3641-4fd9-9688-f39f243dd205 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:13.029729 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff4793-3641-4fd9-9688-f39f243dd205 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:13.030627 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff4793-3641-4fd9-9688-f39f243dd205 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:13.031976 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff4793-3641-4fd9-9688-f39f243dd205 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:13.043764 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-6dff4793-3641-4fd9-9688-f39f243dd205 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 06:10:13.043764 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:10:13.043764 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 06:10:13.043764 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 06:10:13.043764 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 06:10:13.043764 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 06:10:13.043764 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 06:10:13.043764 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:10:13.043764 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 06:10:13.043764 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:10:13.043764 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 06:10:13.043764 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:10:13.043764 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:10:13.043764 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:10:13.043764 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:10:13.043764 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:10:13.043764 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:10:13.045900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:10:13.045900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:10:13.045900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:10:13.045900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:10:13.045900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:10:13.045900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:10:13.045900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 06:10:13.045900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 06:10:13.045900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 06:10:13.045900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 06:10:13.045900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:10:13.045900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:10:13.045900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:10:13.045900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:10:13.045900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:10:13.045900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:10:13.046981 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:10:13.046981 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:10:13.046981 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:10:13.046981 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:10:13.046981 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:10:13.046981 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:10:13.046981 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:10:13.046981 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:10:13.046981 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:10:13.046981 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:10:13.046981 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 06:10:13.046981 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:10:13.046981 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 06:10:13.046981 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:10:13.046981 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 635/1290] 10.222.0.40 () {56 vars in 1117 bytes} [Mon Jul 18 06:10:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 06:10:13.051754 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb099d6-a16d-47da-aeb4-224d8af62a33 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:13.155356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb099d6-a16d-47da-aeb4-224d8af62a33 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:13.190445 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328b515-5e24-4383-9c57-f587ca3a05fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:13.191113 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328b515-5e24-4383-9c57-f587ca3a05fb None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:13.191403 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328b515-5e24-4383-9c57-f587ca3a05fb None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:13.191687 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328b515-5e24-4383-9c57-f587ca3a05fb None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:13.192404 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f328b515-5e24-4383-9c57-f587ca3a05fb None admin] RBAC: Adding query filter params (name=placement) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:10:13.192745 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f328b515-5e24-4383-9c57-f587ca3a05fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:13.194168 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f328b515-5e24-4383-9c57-f587ca3a05fb None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:13.199024 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 656/1291] 10.222.0.40 () {56 vars in 1116 bytes} [Mon Jul 18 06:10:13 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:10:13.204993 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff4793-3641-4fd9-9688-f39f243dd205 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:13.306594 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff4793-3641-4fd9-9688-f39f243dd205 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:13.341751 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7445ba-3f73-4ff4-b98b-3ba5d7706938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:13.342417 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7445ba-3f73-4ff4-b98b-3ba5d7706938 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:13.342705 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7445ba-3f73-4ff4-b98b-3ba5d7706938 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:13.342983 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7445ba-3f73-4ff4-b98b-3ba5d7706938 None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:13.343695 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7445ba-3f73-4ff4-b98b-3ba5d7706938 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:10:13.344003 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7445ba-3f73-4ff4-b98b-3ba5d7706938 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:13.345415 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7445ba-3f73-4ff4-b98b-3ba5d7706938 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:13.350140 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 636/1292] 10.222.0.40 () {56 vars in 1116 bytes} [Mon Jul 18 06:10:13 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:10:15.354262 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0d798-68a3-4494-95a9-4096472a8e22 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:15.354840 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0d798-68a3-4494-95a9-4096472a8e22 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:15.354840 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0d798-68a3-4494-95a9-4096472a8e22 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:15.356192 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 657/1293] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:10:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:10:15.362030 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92862f-82fd-45d1-ae05-198c9cfa515e None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:15.362456 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92862f-82fd-45d1-ae05-198c9cfa515e None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:15.362617 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92862f-82fd-45d1-ae05-198c9cfa515e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:15.379922 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-4e92862f-82fd-45d1-ae05-198c9cfa515e None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:10:15.381195 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4e92862f-82fd-45d1-ae05-198c9cfa515e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:10:15.410407 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:10:15.446651 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 637/1294] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:10:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:10:15.469610 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed0d798-68a3-4494-95a9-4096472a8e22 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:15.501400 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:10:15.571744 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed0d798-68a3-4494-95a9-4096472a8e22 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:15.605572 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f02031-7753-4271-b2d3-9f2b29d6c028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:15.606163 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f02031-7753-4271-b2d3-9f2b29d6c028 None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:15.606446 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f02031-7753-4271-b2d3-9f2b29d6c028 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:15.606817 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f02031-7753-4271-b2d3-9f2b29d6c028 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:15.608207 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f02031-7753-4271-b2d3-9f2b29d6c028 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:15.609655 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f02031-7753-4271-b2d3-9f2b29d6c028 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:15.689372 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 658/1295] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Jul 18 06:10:15 2022] POST /identity/v3/services => generated 246 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:10:17.724687 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7fe1f-0223-4ff1-84aa-28aa50a9bf3d None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:17.725207 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7fe1f-0223-4ff1-84aa-28aa50a9bf3d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:17.725328 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7fe1f-0223-4ff1-84aa-28aa50a9bf3d None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:17.726647 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 638/1296] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:10:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:10:17.732544 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81901846-cd0e-4f7b-9de7-58c1f0d75e78 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:17.733020 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81901846-cd0e-4f7b-9de7-58c1f0d75e78 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:17.733195 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81901846-cd0e-4f7b-9de7-58c1f0d75e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:17.750369 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-81901846-cd0e-4f7b-9de7-58c1f0d75e78 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:10:17.751562 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-81901846-cd0e-4f7b-9de7-58c1f0d75e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:10:17.824785 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 659/1297] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:10:17 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:10:17.847955 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7fe1f-0223-4ff1-84aa-28aa50a9bf3d None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:17.951080 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7fe1f-0223-4ff1-84aa-28aa50a9bf3d None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:17.985418 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa23893-bb01-4bdf-b8d8-f3ce2772924d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:17.986213 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa23893-bb01-4bdf-b8d8-f3ce2772924d None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:17.986555 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa23893-bb01-4bdf-b8d8-f3ce2772924d None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:17.986831 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa23893-bb01-4bdf-b8d8-f3ce2772924d None admin] PATH_INFO: `/v3/services/placement` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:17.987514 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa23893-bb01-4bdf-b8d8-f3ce2772924d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:17.988922 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa23893-bb01-4bdf-b8d8-f3ce2772924d None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:18.000934 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-daa23893-bb01-4bdf-b8d8-f3ce2772924d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 06:10:18.000934 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:10:18.000934 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 06:10:18.000934 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 06:10:18.000934 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 06:10:18.000934 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 06:10:18.000934 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 06:10:18.000934 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:10:18.000934 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 06:10:18.000934 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:10:18.000934 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 06:10:18.000934 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:10:18.000934 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:10:18.000934 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:10:18.000934 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:10:18.000934 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:10:18.000934 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:10:18.002147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:10:18.002147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:10:18.002147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:10:18.002147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:10:18.002147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:10:18.002147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:10:18.002147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 06:10:18.002147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 06:10:18.002147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 06:10:18.002147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 06:10:18.002147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:10:18.002147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:10:18.002147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:10:18.002147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:10:18.002147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:10:18.002147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:10:18.003361 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:10:18.003361 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:10:18.003361 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:10:18.003361 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:10:18.003361 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:10:18.003361 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:10:18.003361 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:10:18.003361 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:10:18.003361 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:10:18.003361 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:10:18.003361 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 06:10:18.003361 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:10:18.003361 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 06:10:18.003361 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:10:18.003361 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 639/1298] 10.222.0.40 () {56 vars in 1117 bytes} [Mon Jul 18 06:10:17 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:10:18.008180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81901846-cd0e-4f7b-9de7-58c1f0d75e78 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:18.026847 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:10:18.107923 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81901846-cd0e-4f7b-9de7-58c1f0d75e78 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:18.141203 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b81bde9-1455-45f5-9b4d-33bcfdaaae98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:18.141867 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81bde9-1455-45f5-9b4d-33bcfdaaae98 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:18.142169 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81bde9-1455-45f5-9b4d-33bcfdaaae98 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:18.142439 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81bde9-1455-45f5-9b4d-33bcfdaaae98 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:18.143146 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b81bde9-1455-45f5-9b4d-33bcfdaaae98 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:10:18.143456 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b81bde9-1455-45f5-9b4d-33bcfdaaae98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:18.144867 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b81bde9-1455-45f5-9b4d-33bcfdaaae98 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:18.149906 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 660/1299] 10.222.0.40 () {56 vars in 1116 bytes} [Mon Jul 18 06:10:18 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:10:18.156704 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa23893-bb01-4bdf-b8d8-f3ce2772924d None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:18.188563 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:10:18.255871 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa23893-bb01-4bdf-b8d8-f3ce2772924d None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:18.290388 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f6b163-9236-421e-983b-3fe49e75cd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:18.291045 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6b163-9236-421e-983b-3fe49e75cd72 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:18.291333 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6b163-9236-421e-983b-3fe49e75cd72 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:18.291611 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6b163-9236-421e-983b-3fe49e75cd72 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:18.292445 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f6b163-9236-421e-983b-3fe49e75cd72 None admin] RBAC: Adding query filter params (interface=public, service_id=edd1746e488747ff8a03c00a724f5777, region_id=RegionOne) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:10:18.293110 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f6b163-9236-421e-983b-3fe49e75cd72 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:18.294449 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f6b163-9236-421e-983b-3fe49e75cd72 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:18.299703 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 640/1300] 10.222.0.40 () {56 vars in 1251 bytes} [Mon Jul 18 06:10:18 2022] GET /identity/v3/endpoints?service_id=edd1746e488747ff8a03c00a724f5777&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:10:18.305717 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b81bde9-1455-45f5-9b4d-33bcfdaaae98 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:18.387078 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:10:18.406533 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b81bde9-1455-45f5-9b4d-33bcfdaaae98 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:18.440289 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5b1582-6bba-40e8-ae7f-cb9a5ccec4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:18.440985 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b1582-6bba-40e8-ae7f-cb9a5ccec4c3 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:18.441275 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b1582-6bba-40e8-ae7f-cb9a5ccec4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:18.441555 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b1582-6bba-40e8-ae7f-cb9a5ccec4c3 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:18.442259 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5b1582-6bba-40e8-ae7f-cb9a5ccec4c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:18.443735 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5b1582-6bba-40e8-ae7f-cb9a5ccec4c3 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:18.451032 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 661/1301] 10.222.0.40 () {56 vars in 1087 bytes} [Mon Jul 18 06:10:18 2022] GET /identity/v3/services => generated 2191 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 06:10:20.515046 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7148e2-3c8a-4a7f-976d-6397ef861d61 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:20.515849 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7148e2-3c8a-4a7f-976d-6397ef861d61 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:20.515849 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7148e2-3c8a-4a7f-976d-6397ef861d61 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:20.517037 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 641/1302] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:10:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:10:20.522574 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5002a46b-81a6-4662-b24f-02cb2856d1ca None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:20.523009 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5002a46b-81a6-4662-b24f-02cb2856d1ca None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:20.524381 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5002a46b-81a6-4662-b24f-02cb2856d1ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:20.539760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-5002a46b-81a6-4662-b24f-02cb2856d1ca None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:10:20.541010 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-5002a46b-81a6-4662-b24f-02cb2856d1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:10:20.607817 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 662/1303] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:10:20 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 06:10:20.631116 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7148e2-3c8a-4a7f-976d-6397ef861d61 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:20.728245 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:10:20.735667 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7148e2-3c8a-4a7f-976d-6397ef861d61 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:20.768871 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f954065-11d1-4fe2-b9a0-8f9573500e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:20.769595 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f954065-11d1-4fe2-b9a0-8f9573500e4d None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:20.769883 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f954065-11d1-4fe2-b9a0-8f9573500e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:20.770162 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f954065-11d1-4fe2-b9a0-8f9573500e4d None admin] PATH_INFO: `/v3/services/placement` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:20.770854 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f954065-11d1-4fe2-b9a0-8f9573500e4d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:20.772223 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f954065-11d1-4fe2-b9a0-8f9573500e4d None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:20.784324 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-6f954065-11d1-4fe2-b9a0-8f9573500e4d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 06:10:20.784324 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:10:20.784324 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 06:10:20.784324 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 06:10:20.784324 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 06:10:20.784324 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 06:10:20.784324 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 06:10:20.784324 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:10:20.784324 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 06:10:20.784324 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:10:20.784324 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 06:10:20.784324 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:10:20.784324 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 06:10:20.784324 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 06:10:20.784324 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 06:10:20.784324 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 06:10:20.784324 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 06:10:20.785545 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 06:10:20.785545 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 06:10:20.785545 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 06:10:20.785545 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 06:10:20.785545 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 06:10:20.785545 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 06:10:20.785545 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 06:10:20.785545 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 06:10:20.785545 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 06:10:20.785545 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 06:10:20.785545 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 06:10:20.785545 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 06:10:20.785545 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 06:10:20.785545 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 06:10:20.785545 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 06:10:20.785545 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 06:10:20.786598 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 06:10:20.786598 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 18 06:10:20.786598 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 06:10:20.786598 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 18 06:10:20.786598 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 06:10:20.786598 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 06:10:20.786598 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 06:10:20.786598 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 18 06:10:20.786598 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 06:10:20.786598 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 18 06:10:20.786598 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 06:10:20.786598 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 06:10:20.786598 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 06:10:20.786598 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 18 06:10:20.786598 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 642/1304] 10.222.0.40 () {56 vars in 1117 bytes} [Mon Jul 18 06:10:20 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 06:10:20.791735 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5002a46b-81a6-4662-b24f-02cb2856d1ca None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:20.870093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:10:20.895573 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5002a46b-81a6-4662-b24f-02cb2856d1ca None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:20.929197 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b6f00a-de27-40d6-9bfb-b43913388b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:20.929866 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6f00a-de27-40d6-9bfb-b43913388b6e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:20.930153 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6f00a-de27-40d6-9bfb-b43913388b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:20.930435 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6f00a-de27-40d6-9bfb-b43913388b6e None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:20.931145 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b6f00a-de27-40d6-9bfb-b43913388b6e None admin] RBAC: Adding query filter params (name=placement) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:10:20.931456 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b6f00a-de27-40d6-9bfb-b43913388b6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:20.932891 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b6f00a-de27-40d6-9bfb-b43913388b6e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:20.938567 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 663/1305] 10.222.0.40 () {56 vars in 1116 bytes} [Mon Jul 18 06:10:20 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:10:20.944954 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f954065-11d1-4fe2-b9a0-8f9573500e4d None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:10:21.046138 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f954065-11d1-4fe2-b9a0-8f9573500e4d None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:10:21.082182 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6354cd9c-f7e2-4de6-adc4-6fc81181e928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:10:21.082842 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354cd9c-f7e2-4de6-adc4-6fc81181e928 None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:10:21.083128 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354cd9c-f7e2-4de6-adc4-6fc81181e928 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:10:21.083408 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354cd9c-f7e2-4de6-adc4-6fc81181e928 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:10:21.084165 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6354cd9c-f7e2-4de6-adc4-6fc81181e928 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:10:21.085574 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6354cd9c-f7e2-4de6-adc4-6fc81181e928 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:10:21.253458 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 643/1306] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Jul 18 06:10:20 2022] POST /identity/v3/endpoints => generated 331 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:10:30.970898 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=30 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:10:31.247007 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:17:25.466690 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e572b0f-a92f-4fbf-904e-10ec378e794d None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:17:25.467294 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e572b0f-a92f-4fbf-904e-10ec378e794d None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:17:25.467294 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e572b0f-a92f-4fbf-904e-10ec378e794d None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:17:25.474972 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26aaa10-129b-4ecb-a84c-3c80a67d0955 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:17:25.475481 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26aaa10-129b-4ecb-a84c-3c80a67d0955 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:17:25.475578 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26aaa10-129b-4ecb-a84c-3c80a67d0955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:17:25.479811 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 664/1307] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:17:25 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:17:25.575883 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-a26aaa10-129b-4ecb-a84c-3c80a67d0955 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:17:25.577228 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a26aaa10-129b-4ecb-a84c-3c80a67d0955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:17:25.721882 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 644/1308] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:17:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:17:25.766954 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c53c2d-4d9f-49cc-b847-30c2609137d3 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:17:25.767248 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c53c2d-4d9f-49cc-b847-30c2609137d3 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:17:25.767431 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c53c2d-4d9f-49cc-b847-30c2609137d3 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:17:25.769102 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 665/1309] 10.222.0.40 () {54 vars in 965 bytes} [Mon Jul 18 06:17:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:17:25.776396 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ad305-26c1-4cb3-88d4-16910f123295 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:17:25.776719 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ad305-26c1-4cb3-88d4-16910f123295 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:17:25.777091 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ad305-26c1-4cb3-88d4-16910f123295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:17:25.865962 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-e36ad305-26c1-4cb3-88d4-16910f123295 None None] MFA Rules not processed for user `9824a96e2bcd4a16bfc6fe6bf4ea1ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:17:25.867177 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e36ad305-26c1-4cb3-88d4-16910f123295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:17:25.891443 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:17:25.956146 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 645/1310] 10.222.0.40 () {58 vars in 1062 bytes} [Mon Jul 18 06:17:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:17:25.967338 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c53c2d-4d9f-49cc-b847-30c2609137d3 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:17:25.969541 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-78c53c2d-4d9f-49cc-b847-30c2609137d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:17:26.131968 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c53c2d-4d9f-49cc-b847-30c2609137d3 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:17:26.225678 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa333930-c65a-4731-a5ef-6a169558c6d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:17:26.226252 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa333930-c65a-4731-a5ef-6a169558c6d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:17:26.226546 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa333930-c65a-4731-a5ef-6a169558c6d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:17:26.226829 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa333930-c65a-4731-a5ef-6a169558c6d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:17:26.228579 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-aa333930-c65a-4731-a5ef-6a169558c6d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:17:26.310459 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:17:26.384060 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa333930-c65a-4731-a5ef-6a169558c6d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:17:26.386194 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa333930-c65a-4731-a5ef-6a169558c6d2 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:17:26.415443 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 666/1311] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:17:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:17:28.779501 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c13c21c-3946-4029-8b00-84e2006ebc79 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:17:28.780089 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c13c21c-3946-4029-8b00-84e2006ebc79 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:17:28.780089 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c13c21c-3946-4029-8b00-84e2006ebc79 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:17:28.781551 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 646/1312] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:17:28.787564 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fec43f-4e32-4c3f-932c-59c29d13b979 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:17:28.788124 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fec43f-4e32-4c3f-932c-59c29d13b979 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:17:28.788253 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fec43f-4e32-4c3f-932c-59c29d13b979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:17:28.871458 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-77fec43f-4e32-4c3f-932c-59c29d13b979 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:17:28.872638 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-77fec43f-4e32-4c3f-932c-59c29d13b979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:17:28.977870 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 667/1313] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:17:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:17:29.016906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c13c21c-3946-4029-8b00-84e2006ebc79 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:17:29.018468 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8c13c21c-3946-4029-8b00-84e2006ebc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:17:29.128301 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c13c21c-3946-4029-8b00-84e2006ebc79 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:17:29.208128 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:17:29.211033 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e3e864-174f-479b-bc40-be04a39a9c44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:17:29.212373 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3e864-174f-479b-bc40-be04a39a9c44 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:17:29.212725 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3e864-174f-479b-bc40-be04a39a9c44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:17:29.212978 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3e864-174f-479b-bc40-be04a39a9c44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:17:29.214682 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-c6e3e864-174f-479b-bc40-be04a39a9c44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:17:29.337029 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e3e864-174f-479b-bc40-be04a39a9c44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:17:29.338688 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e3e864-174f-479b-bc40-be04a39a9c44 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:17:29.357598 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 647/1314] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:17:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:17:31.926089 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b9830-5947-4d48-a08d-924c015e4203 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:17:31.926590 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b9830-5947-4d48-a08d-924c015e4203 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:17:31.926726 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b9830-5947-4d48-a08d-924c015e4203 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:17:31.928135 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 668/1315] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:17:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:17:31.934379 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b3839-0372-44d7-8834-6180c9677309 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:17:31.934859 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b3839-0372-44d7-8834-6180c9677309 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:17:31.934956 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b3839-0372-44d7-8834-6180c9677309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:17:32.004731 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-3a1b3839-0372-44d7-8834-6180c9677309 None None] MFA Rules not processed for user `ab532fe0550b4d79a1bc56aff3a426f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:17:32.005974 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-3a1b3839-0372-44d7-8834-6180c9677309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:17:32.060109 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:17:32.107139 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 648/1316] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:17:31 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:17:32.147965 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b9830-5947-4d48-a08d-924c015e4203 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:17:32.149586 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-431b9830-5947-4d48-a08d-924c015e4203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:17:32.159386 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:17:32.256891 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b9830-5947-4d48-a08d-924c015e4203 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:17:32.333281 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563718ea-7063-4f7f-aaeb-67bf0c887e6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:17:32.333927 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563718ea-7063-4f7f-aaeb-67bf0c887e6e service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:17:32.334210 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563718ea-7063-4f7f-aaeb-67bf0c887e6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:17:32.334483 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563718ea-7063-4f7f-aaeb-67bf0c887e6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:17:32.336720 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-563718ea-7063-4f7f-aaeb-67bf0c887e6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:17:32.399860 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:17:32.448479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563718ea-7063-4f7f-aaeb-67bf0c887e6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:17:32.449927 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563718ea-7063-4f7f-aaeb-67bf0c887e6e service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:17:32.468985 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 669/1317] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:17:32 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:17:35.346358 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67c7a3-4564-4605-b18e-4c4a97953464 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:17:35.346913 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67c7a3-4564-4605-b18e-4c4a97953464 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:17:35.346913 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67c7a3-4564-4605-b18e-4c4a97953464 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:17:35.348330 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 649/1318] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:17:35.354314 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46ce54-8bd3-4cfd-b4b6-04bbaffdbba7 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:17:35.354716 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46ce54-8bd3-4cfd-b4b6-04bbaffdbba7 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:17:35.354878 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46ce54-8bd3-4cfd-b4b6-04bbaffdbba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:17:35.421430 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-6e46ce54-8bd3-4cfd-b4b6-04bbaffdbba7 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:17:35.422641 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-6e46ce54-8bd3-4cfd-b4b6-04bbaffdbba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:17:35.522469 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 670/1319] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:17:35 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:17:35.561495 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a67c7a3-4564-4605-b18e-4c4a97953464 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:17:35.563023 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-9a67c7a3-4564-4605-b18e-4c4a97953464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:17:35.669837 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a67c7a3-4564-4605-b18e-4c4a97953464 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:17:35.750304 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c64a08-4b6a-41c7-b3b1-d41a6eb66a0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:17:35.758740 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c64a08-4b6a-41c7-b3b1-d41a6eb66a0b service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:17:35.758740 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c64a08-4b6a-41c7-b3b1-d41a6eb66a0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:17:35.758740 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c64a08-4b6a-41c7-b3b1-d41a6eb66a0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:17:35.758740 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e1c64a08-4b6a-41c7-b3b1-d41a6eb66a0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:17:35.762184 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:17:35.870390 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c64a08-4b6a-41c7-b3b1-d41a6eb66a0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:17:35.871748 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c64a08-4b6a-41c7-b3b1-d41a6eb66a0b service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:17:35.892523 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 650/1320] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:17:35 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:17:39.002261 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff7b61-73d8-4870-9f54-0535529c709a None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:17:39.002824 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff7b61-73d8-4870-9f54-0535529c709a None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:17:39.002824 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff7b61-73d8-4870-9f54-0535529c709a None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:17:39.004192 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 671/1321] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:17:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:17:39.009929 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19447567-9ce2-4e73-8b5f-ad6a86c7d065 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:17:39.010409 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19447567-9ce2-4e73-8b5f-ad6a86c7d065 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:17:39.010505 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19447567-9ce2-4e73-8b5f-ad6a86c7d065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:17:39.083760 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-19447567-9ce2-4e73-8b5f-ad6a86c7d065 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:17:39.085000 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-19447567-9ce2-4e73-8b5f-ad6a86c7d065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:17:39.145804 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:17:39.187732 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 651/1322] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:17:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:17:39.224026 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff7b61-73d8-4870-9f54-0535529c709a None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:17:39.225636 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-0fff7b61-73d8-4870-9f54-0535529c709a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:17:39.247413 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:17:39.345812 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff7b61-73d8-4870-9f54-0535529c709a None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:17:39.434425 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f9afd5-28c4-405f-a9ba-a182552a2c75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:17:39.435105 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9afd5-28c4-405f-a9ba-a182552a2c75 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:17:39.435395 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9afd5-28c4-405f-a9ba-a182552a2c75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:17:39.435676 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9afd5-28c4-405f-a9ba-a182552a2c75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:17:39.437523 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-a5f9afd5-28c4-405f-a9ba-a182552a2c75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:17:39.525183 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:17:39.571030 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f9afd5-28c4-405f-a9ba-a182552a2c75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:17:39.572528 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f9afd5-28c4-405f-a9ba-a182552a2c75 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:17:39.591873 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 672/1323] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:17:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:17:41.701196 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b96eb-4853-4ec0-b3f3-5cb169164801 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:17:41.701739 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b96eb-4853-4ec0-b3f3-5cb169164801 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:17:41.701739 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b96eb-4853-4ec0-b3f3-5cb169164801 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:17:41.703404 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 652/1324] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:17:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:17:41.709377 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c1ac1-42a7-4a0a-be03-94930262b04e None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:17:41.710036 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c1ac1-42a7-4a0a-be03-94930262b04e None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:17:41.710036 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c1ac1-42a7-4a0a-be03-94930262b04e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:17:41.785952 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-d22c1ac1-42a7-4a0a-be03-94930262b04e None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:17:41.787162 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d22c1ac1-42a7-4a0a-be03-94930262b04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:17:41.892559 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 673/1325] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:17:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:17:41.929174 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b96eb-4853-4ec0-b3f3-5cb169164801 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:17:41.931788 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-194b96eb-4853-4ec0-b3f3-5cb169164801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:17:42.043768 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b96eb-4853-4ec0-b3f3-5cb169164801 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:17:42.124951 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1124af-56b9-4e50-b6a8-4752ed47c337 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:17:42.125643 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1124af-56b9-4e50-b6a8-4752ed47c337 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:17:42.125924 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1124af-56b9-4e50-b6a8-4752ed47c337 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:17:42.126195 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1124af-56b9-4e50-b6a8-4752ed47c337 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:17:42.127904 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7e1124af-56b9-4e50-b6a8-4752ed47c337 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:17:42.146580 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:17:42.250346 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1124af-56b9-4e50-b6a8-4752ed47c337 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:17:42.251600 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1124af-56b9-4e50-b6a8-4752ed47c337 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:17:42.271472 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 653/1326] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:17:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:17:44.675801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681bc26-e5f2-477f-8a6c-ba5b5eae5b22 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:17:44.676374 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681bc26-e5f2-477f-8a6c-ba5b5eae5b22 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:17:44.676572 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681bc26-e5f2-477f-8a6c-ba5b5eae5b22 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:17:44.677898 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 674/1327] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:17:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:17:44.683598 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd68b338-09d7-4907-9ab8-fecf0b0d4c8b None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:17:44.684081 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd68b338-09d7-4907-9ab8-fecf0b0d4c8b None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:17:44.684160 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd68b338-09d7-4907-9ab8-fecf0b0d4c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:17:44.756746 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-fd68b338-09d7-4907-9ab8-fecf0b0d4c8b None None] MFA Rules not processed for user `ab532fe0550b4d79a1bc56aff3a426f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:17:44.758001 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-fd68b338-09d7-4907-9ab8-fecf0b0d4c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:17:44.831863 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:17:44.873022 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 654/1328] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:17:44 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:17:44.931554 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f5f17-2205-46ac-bcf5-1e2c6c5e8dd9 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:17:44.931747 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f5f17-2205-46ac-bcf5-1e2c6c5e8dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:17:44.932023 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f5f17-2205-46ac-bcf5-1e2c6c5e8dd9 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:17:44.933468 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 675/1329] 10.222.0.40 () {54 vars in 965 bytes} [Mon Jul 18 06:17:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:17:44.939738 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdbee0a-f62d-4018-9f99-89ccda8419e9 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:17:44.940030 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdbee0a-f62d-4018-9f99-89ccda8419e9 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:17:44.940319 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdbee0a-f62d-4018-9f99-89ccda8419e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:17:45.024683 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-2bdbee0a-f62d-4018-9f99-89ccda8419e9 None None] MFA Rules not processed for user `9824a96e2bcd4a16bfc6fe6bf4ea1ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:17:45.026019 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-2bdbee0a-f62d-4018-9f99-89ccda8419e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:17:45.112137 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 655/1330] 10.222.0.40 () {58 vars in 1062 bytes} [Mon Jul 18 06:17:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:17:45.121689 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f5f17-2205-46ac-bcf5-1e2c6c5e8dd9 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:17:45.123215 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-b47f5f17-2205-46ac-bcf5-1e2c6c5e8dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:17:45.156015 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:17:45.240074 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f5f17-2205-46ac-bcf5-1e2c6c5e8dd9 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:17:45.321552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475f4f99-6ba6-4484-85c3-783b6327c597 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:17:45.322223 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f4f99-6ba6-4484-85c3-783b6327c597 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:17:45.322513 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f4f99-6ba6-4484-85c3-783b6327c597 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:17:45.322791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f4f99-6ba6-4484-85c3-783b6327c597 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:17:45.324544 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-475f4f99-6ba6-4484-85c3-783b6327c597 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:17:45.411035 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:17:45.448615 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475f4f99-6ba6-4484-85c3-783b6327c597 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:17:45.450011 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475f4f99-6ba6-4484-85c3-783b6327c597 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:17:45.472530 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 676/1331] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:17:45 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:17:48.365948 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871759a9-e57d-405d-b881-7c7be81a736a None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:17:48.367102 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871759a9-e57d-405d-b881-7c7be81a736a None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:17:48.367102 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871759a9-e57d-405d-b881-7c7be81a736a None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:17:48.368458 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 656/1332] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:17:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:17:48.373848 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79499b92-9413-4924-b42f-8eb2ab085007 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:17:48.374337 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79499b92-9413-4924-b42f-8eb2ab085007 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:17:48.374446 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79499b92-9413-4924-b42f-8eb2ab085007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:17:48.441496 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-79499b92-9413-4924-b42f-8eb2ab085007 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:17:48.442702 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-79499b92-9413-4924-b42f-8eb2ab085007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:17:48.547519 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 677/1333] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:17:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:17:48.584045 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871759a9-e57d-405d-b881-7c7be81a736a None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:17:48.586211 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-871759a9-e57d-405d-b881-7c7be81a736a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:17:48.658380 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:17:48.693500 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871759a9-e57d-405d-b881-7c7be81a736a None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:17:48.771012 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6cb1e5-1bf3-4f08-9482-2dbea885587d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:17:48.771664 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cb1e5-1bf3-4f08-9482-2dbea885587d service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:17:48.771953 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cb1e5-1bf3-4f08-9482-2dbea885587d service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:17:48.772233 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cb1e5-1bf3-4f08-9482-2dbea885587d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:17:48.773952 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-cc6cb1e5-1bf3-4f08-9482-2dbea885587d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:17:48.884218 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6cb1e5-1bf3-4f08-9482-2dbea885587d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:17:48.885684 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6cb1e5-1bf3-4f08-9482-2dbea885587d service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:17:48.895853 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:17:48.904078 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 657/1334] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:17:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:17:50.955859 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b871f-55a5-4aa0-8c4f-5e8e5cfd6f12 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:17:50.956379 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b871f-55a5-4aa0-8c4f-5e8e5cfd6f12 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:17:50.956379 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b871f-55a5-4aa0-8c4f-5e8e5cfd6f12 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:17:50.957747 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 678/1335] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:17:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:17:50.963231 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc2b91-9e62-4735-9cd9-b08086edaa94 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:17:50.963702 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc2b91-9e62-4735-9cd9-b08086edaa94 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:17:50.963826 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc2b91-9e62-4735-9cd9-b08086edaa94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:17:51.029797 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-f4bc2b91-9e62-4735-9cd9-b08086edaa94 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:17:51.031020 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f4bc2b91-9e62-4735-9cd9-b08086edaa94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:17:51.128506 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 658/1336] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:17:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:17:51.169313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b871f-55a5-4aa0-8c4f-5e8e5cfd6f12 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:17:51.170919 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c77b871f-55a5-4aa0-8c4f-5e8e5cfd6f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:17:51.212382 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:17:51.279425 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b871f-55a5-4aa0-8c4f-5e8e5cfd6f12 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:17:51.358077 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a6ac26-6189-4b09-8160-884449b0cd84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:17:51.358732 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6ac26-6189-4b09-8160-884449b0cd84 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:17:51.359016 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6ac26-6189-4b09-8160-884449b0cd84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:17:51.359293 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6ac26-6189-4b09-8160-884449b0cd84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:17:51.360968 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-74a6ac26-6189-4b09-8160-884449b0cd84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:17:51.447444 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:17:51.473192 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a6ac26-6189-4b09-8160-884449b0cd84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:17:51.474562 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a6ac26-6189-4b09-8160-884449b0cd84 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:17:51.493568 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 679/1337] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:17:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:17:54.146760 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37146dd9-7c9d-49ef-ad48-90262b5271db None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:17:54.147299 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37146dd9-7c9d-49ef-ad48-90262b5271db None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:17:54.147395 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37146dd9-7c9d-49ef-ad48-90262b5271db None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:17:54.148747 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 659/1338] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:17:54.154514 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8c587-d02e-434c-ba37-8074fbc3b9b9 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:17:54.154992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8c587-d02e-434c-ba37-8074fbc3b9b9 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:17:54.155092 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8c587-d02e-434c-ba37-8074fbc3b9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:17:54.221326 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-8ba8c587-d02e-434c-ba37-8074fbc3b9b9 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:17:54.222556 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-8ba8c587-d02e-434c-ba37-8074fbc3b9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:17:54.323402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 680/1339] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:17:54 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:17:54.360632 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37146dd9-7c9d-49ef-ad48-90262b5271db None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:17:54.362192 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-37146dd9-7c9d-49ef-ad48-90262b5271db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:17:54.439157 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:17:54.467349 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37146dd9-7c9d-49ef-ad48-90262b5271db None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:17:54.541935 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b9a13d-f8a7-4e5d-8d79-e2ed28fd29f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:17:54.542592 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9a13d-f8a7-4e5d-8d79-e2ed28fd29f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:17:54.542878 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9a13d-f8a7-4e5d-8d79-e2ed28fd29f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:17:54.543157 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9a13d-f8a7-4e5d-8d79-e2ed28fd29f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:17:54.544846 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a0b9a13d-f8a7-4e5d-8d79-e2ed28fd29f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:17:54.657995 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b9a13d-f8a7-4e5d-8d79-e2ed28fd29f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:17:54.659427 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b9a13d-f8a7-4e5d-8d79-e2ed28fd29f5 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:17:54.678264 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 660/1340] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:17:54 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:17:59.223610 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12433d2a-56d5-4468-b210-af300f6c59de None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:17:59.224222 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12433d2a-56d5-4468-b210-af300f6c59de None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:17:59.224222 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12433d2a-56d5-4468-b210-af300f6c59de None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:17:59.228652 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 681/1341] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:17:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:17:59.232258 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e50bb0-5520-41b8-b9c2-73f9f5685b27 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:17:59.232772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e50bb0-5520-41b8-b9c2-73f9f5685b27 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:17:59.232983 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e50bb0-5520-41b8-b9c2-73f9f5685b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:17:59.237334 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:17:59.321257 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-a9e50bb0-5520-41b8-b9c2-73f9f5685b27 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:17:59.322805 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a9e50bb0-5520-41b8-b9c2-73f9f5685b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:17:59.439934 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 661/1342] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:17:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:17:59.479864 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12433d2a-56d5-4468-b210-af300f6c59de None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:17:59.481486 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-12433d2a-56d5-4468-b210-af300f6c59de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:17:59.539973 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:17:59.599178 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12433d2a-56d5-4468-b210-af300f6c59de None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:17:59.680519 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0986ab-2500-4432-8fd1-c51f0d87976e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:17:59.683735 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0986ab-2500-4432-8fd1-c51f0d87976e service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:17:59.683735 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0986ab-2500-4432-8fd1-c51f0d87976e service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:17:59.683735 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0986ab-2500-4432-8fd1-c51f0d87976e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:17:59.683735 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-9e0986ab-2500-4432-8fd1-c51f0d87976e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:17:59.783360 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:17:59.801433 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0986ab-2500-4432-8fd1-c51f0d87976e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:17:59.802885 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0986ab-2500-4432-8fd1-c51f0d87976e service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:17:59.822968 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 682/1343] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:17:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:18:02.905331 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b24b8-d211-4b4b-b3e3-3148b79dc497 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:02.905862 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b24b8-d211-4b4b-b3e3-3148b79dc497 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:02.905951 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b24b8-d211-4b4b-b3e3-3148b79dc497 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:02.907333 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 662/1344] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:18:02.913594 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea064588-d779-45ad-b38c-63039a00aad3 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:02.914062 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea064588-d779-45ad-b38c-63039a00aad3 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:02.914347 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea064588-d779-45ad-b38c-63039a00aad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:02.983452 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-ea064588-d779-45ad-b38c-63039a00aad3 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:18:02.984605 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ea064588-d779-45ad-b38c-63039a00aad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:03.086853 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 683/1345] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:18:02 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:18:03.125304 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200b24b8-d211-4b4b-b3e3-3148b79dc497 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:18:03.126833 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-200b24b8-d211-4b4b-b3e3-3148b79dc497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:03.217444 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:03.234523 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200b24b8-d211-4b4b-b3e3-3148b79dc497 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:18:03.315205 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a5a51-264a-4b29-9e83-868b6c4702a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:18:03.315861 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a5a51-264a-4b29-9e83-868b6c4702a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:03.316154 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a5a51-264a-4b29-9e83-868b6c4702a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:03.316459 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a5a51-264a-4b29-9e83-868b6c4702a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:03.318145 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-599a5a51-264a-4b29-9e83-868b6c4702a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:03.440550 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599a5a51-264a-4b29-9e83-868b6c4702a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:18:03.441987 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599a5a51-264a-4b29-9e83-868b6c4702a0 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:18:03.463566 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 663/1346] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:18:03 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 06:18:08.167911 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd75491-51ba-40c7-af48-651108cbf0be None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:08.168886 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd75491-51ba-40c7-af48-651108cbf0be None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:08.168886 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd75491-51ba-40c7-af48-651108cbf0be None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:08.170351 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 684/1347] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:18:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:18:08.175322 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07769770-593d-4b3a-b880-5edf47280fce None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:08.175825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07769770-593d-4b3a-b880-5edf47280fce None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:08.175948 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07769770-593d-4b3a-b880-5edf47280fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:08.191702 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:08.248006 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-07769770-593d-4b3a-b880-5edf47280fce None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:18:08.249262 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-07769770-593d-4b3a-b880-5edf47280fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:08.358401 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 664/1348] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:18:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:18:08.399469 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd75491-51ba-40c7-af48-651108cbf0be None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:18:08.401175 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-0dd75491-51ba-40c7-af48-651108cbf0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:08.464369 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:08.514181 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd75491-51ba-40c7-af48-651108cbf0be None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:18:08.594653 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1cede-0fe8-4736-afd1-1cdc3dd57e2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:18:08.595318 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1cede-0fe8-4736-afd1-1cdc3dd57e2b service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:08.595608 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1cede-0fe8-4736-afd1-1cdc3dd57e2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:08.595891 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1cede-0fe8-4736-afd1-1cdc3dd57e2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:08.597615 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-23f1cede-0fe8-4736-afd1-1cdc3dd57e2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:08.714136 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:08.718086 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f1cede-0fe8-4736-afd1-1cdc3dd57e2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:18:08.719518 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f1cede-0fe8-4736-afd1-1cdc3dd57e2b service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:18:08.738892 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 685/1349] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:18:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:18:13.843568 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8312d-3d94-46b7-9140-b111dd623e60 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:13.844353 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8312d-3d94-46b7-9140-b111dd623e60 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:13.844353 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8312d-3d94-46b7-9140-b111dd623e60 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:13.845691 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 665/1350] 10.222.0.40 () {54 vars in 921 bytes} [Mon Jul 18 06:18:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:18:13.856118 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c2f72-20e6-4362-b767-1e5a3c4376e1 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:13.857996 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c2f72-20e6-4362-b767-1e5a3c4376e1 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:13.858816 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c2f72-20e6-4362-b767-1e5a3c4376e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:14.018968 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-b39c2f72-20e6-4362-b767-1e5a3c4376e1 None None] MFA Rules not processed for user `a3560534427544cdabfd0ba702a0a165`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:18:14.021921 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-b39c2f72-20e6-4362-b767-1e5a3c4376e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:14.215974 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 686/1351] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Jul 18 06:18:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:18:18.339956 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c31f11-30b5-4d59-9357-b8535eb2c1b1 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:18.340918 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c31f11-30b5-4d59-9357-b8535eb2c1b1 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:18.341188 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c31f11-30b5-4d59-9357-b8535eb2c1b1 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:18.342731 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 666/1352] 10.222.0.40 () {54 vars in 921 bytes} [Mon Jul 18 06:18:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:18:18.347887 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91221dc3-dc83-4246-98bf-47305294e60c None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:18.348343 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91221dc3-dc83-4246-98bf-47305294e60c None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:18.348685 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91221dc3-dc83-4246-98bf-47305294e60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:18.356293 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=59 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:18.408486 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:18.435818 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-91221dc3-dc83-4246-98bf-47305294e60c None None] MFA Rules not processed for user `a3560534427544cdabfd0ba702a0a165`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:18:18.437212 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-91221dc3-dc83-4246-98bf-47305294e60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:18.562929 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 687/1353] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Jul 18 06:18:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:18:19.950175 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49cb2ad-e5d2-4196-b2a9-78e2ace785c9 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:19.950733 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49cb2ad-e5d2-4196-b2a9-78e2ace785c9 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:19.950733 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49cb2ad-e5d2-4196-b2a9-78e2ace785c9 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:19.952105 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 667/1354] 10.222.0.40 () {54 vars in 921 bytes} [Mon Jul 18 06:18:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:18:19.957277 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6454958-5482-4052-ae23-2dc73426d396 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:19.957712 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6454958-5482-4052-ae23-2dc73426d396 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:19.958125 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6454958-5482-4052-ae23-2dc73426d396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:20.040728 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-c6454958-5482-4052-ae23-2dc73426d396 None None] MFA Rules not processed for user `a3560534427544cdabfd0ba702a0a165`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:18:20.041943 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c6454958-5482-4052-ae23-2dc73426d396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:20.086531 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:20.164156 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 688/1355] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Jul 18 06:18:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:18:23.478473 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082cc2a1-bcc7-4371-82c4-58c010bbd74f None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:23.479587 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082cc2a1-bcc7-4371-82c4-58c010bbd74f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:23.480237 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082cc2a1-bcc7-4371-82c4-58c010bbd74f None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:23.486853 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 668/1356] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:18:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:18:23.488767 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c50b868-e3fd-492f-8fdd-dc9069a2be8c None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:23.490877 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c50b868-e3fd-492f-8fdd-dc9069a2be8c None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:23.491533 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c50b868-e3fd-492f-8fdd-dc9069a2be8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:23.574840 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-3c50b868-e3fd-492f-8fdd-dc9069a2be8c None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:18:23.576306 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-3c50b868-e3fd-492f-8fdd-dc9069a2be8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:23.704968 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 689/1357] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:18:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:18:23.762884 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2871565-5126-4e62-a873-57aa07a604a9 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:23.763554 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2871565-5126-4e62-a873-57aa07a604a9 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:23.763937 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2871565-5126-4e62-a873-57aa07a604a9 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:23.765672 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 669/1358] 10.222.0.40 () {54 vars in 962 bytes} [Mon Jul 18 06:18:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:18:23.775009 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7af04-4313-433b-be5b-d065fe09db1e None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:23.781367 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7af04-4313-433b-be5b-d065fe09db1e None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:23.781367 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7af04-4313-433b-be5b-d065fe09db1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:23.797840 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:23.898763 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-c1e7af04-4313-433b-be5b-d065fe09db1e None None] MFA Rules not processed for user `f92d948810ea4241810473506b654f7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:18:23.900277 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c1e7af04-4313-433b-be5b-d065fe09db1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:24.029298 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 690/1359] 10.222.0.40 () {58 vars in 1059 bytes} [Mon Jul 18 06:18:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:18:24.038377 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2871565-5126-4e62-a873-57aa07a604a9 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:18:24.039888 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-c2871565-5126-4e62-a873-57aa07a604a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:24.198407 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2871565-5126-4e62-a873-57aa07a604a9 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:18:24.283913 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:24.336714 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4639e581-fca6-4559-a57f-d73113539b49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:18:24.337549 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639e581-fca6-4559-a57f-d73113539b49 service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:24.337930 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639e581-fca6-4559-a57f-d73113539b49 service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:24.338306 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639e581-fca6-4559-a57f-d73113539b49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:24.340066 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4639e581-fca6-4559-a57f-d73113539b49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:24.450077 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4639e581-fca6-4559-a57f-d73113539b49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:18:24.451861 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4639e581-fca6-4559-a57f-d73113539b49 service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:18:24.471813 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 670/1360] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:18:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:18:26.756590 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df3f8f-bcc9-4ef8-b280-0ae6d9289011 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:26.757182 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df3f8f-bcc9-4ef8-b280-0ae6d9289011 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:26.757182 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df3f8f-bcc9-4ef8-b280-0ae6d9289011 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:26.758939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 691/1361] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:18:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:18:26.765147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823dfd9-8547-466f-b661-c8878dad19b4 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:26.765661 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823dfd9-8547-466f-b661-c8878dad19b4 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:26.765763 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823dfd9-8547-466f-b661-c8878dad19b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:26.837156 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-1823dfd9-8547-466f-b661-c8878dad19b4 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:18:26.838288 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-1823dfd9-8547-466f-b661-c8878dad19b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:26.878863 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:26.952592 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 671/1362] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:18:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:18:26.998877 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df3f8f-bcc9-4ef8-b280-0ae6d9289011 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:18:27.000367 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-29df3f8f-bcc9-4ef8-b280-0ae6d9289011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:27.082397 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:27.183989 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df3f8f-bcc9-4ef8-b280-0ae6d9289011 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:18:27.357538 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50c82d5-cebc-48a7-a467-df1deafce232 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:18:27.358226 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c82d5-cebc-48a7-a467-df1deafce232 service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:27.358520 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c82d5-cebc-48a7-a467-df1deafce232 service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:27.358806 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c82d5-cebc-48a7-a467-df1deafce232 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:27.360551 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-a50c82d5-cebc-48a7-a467-df1deafce232 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:27.388785 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:27.466445 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50c82d5-cebc-48a7-a467-df1deafce232 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:18:27.468561 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50c82d5-cebc-48a7-a467-df1deafce232 service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:18:27.492762 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 692/1363] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:18:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:18:30.053751 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc07ef8-54d3-421c-b22b-c88cc2b7cd6f None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:30.054260 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc07ef8-54d3-421c-b22b-c88cc2b7cd6f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:30.054260 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc07ef8-54d3-421c-b22b-c88cc2b7cd6f None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:30.055616 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 672/1364] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:18:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:18:30.061212 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c5af7-b9d7-4c14-b3fb-9468687b7c90 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:30.061740 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c5af7-b9d7-4c14-b3fb-9468687b7c90 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:30.061834 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c5af7-b9d7-4c14-b3fb-9468687b7c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:30.136319 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-780c5af7-b9d7-4c14-b3fb-9468687b7c90 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:18:30.137604 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-780c5af7-b9d7-4c14-b3fb-9468687b7c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:30.243534 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:30.248464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 693/1365] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:18:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:18:30.300091 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db334a9a-27cd-47ba-a6aa-145dae283d65 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:30.300556 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db334a9a-27cd-47ba-a6aa-145dae283d65 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:30.301333 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db334a9a-27cd-47ba-a6aa-145dae283d65 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:30.303383 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 673/1366] 10.222.0.40 () {54 vars in 962 bytes} [Mon Jul 18 06:18:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:18:30.313475 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6baabdb-f0d8-4452-88f8-489fc7cd3363 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:30.313475 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6baabdb-f0d8-4452-88f8-489fc7cd3363 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:30.313475 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6baabdb-f0d8-4452-88f8-489fc7cd3363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:30.397853 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-c6baabdb-f0d8-4452-88f8-489fc7cd3363 None None] MFA Rules not processed for user `f92d948810ea4241810473506b654f7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:18:30.399071 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c6baabdb-f0d8-4452-88f8-489fc7cd3363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:30.545059 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 694/1367] 10.222.0.40 () {58 vars in 1059 bytes} [Mon Jul 18 06:18:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:18:30.553365 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db334a9a-27cd-47ba-a6aa-145dae283d65 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:18:30.555340 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-db334a9a-27cd-47ba-a6aa-145dae283d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:30.747361 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db334a9a-27cd-47ba-a6aa-145dae283d65 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:18:30.779691 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:30.920836 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79edc4-f7e8-47b0-af5f-ab53a0b915db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:18:30.921464 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79edc4-f7e8-47b0-af5f-ab53a0b915db service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:30.921748 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79edc4-f7e8-47b0-af5f-ab53a0b915db service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:30.922021 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79edc4-f7e8-47b0-af5f-ab53a0b915db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:30.923705 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-1a79edc4-f7e8-47b0-af5f-ab53a0b915db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:31.033317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79edc4-f7e8-47b0-af5f-ab53a0b915db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:18:31.034847 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79edc4-f7e8-47b0-af5f-ab53a0b915db service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:18:31.056681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 674/1368] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:18:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:18:31.837845 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bda54-21c9-4389-bdd5-4beebbda940c None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:31.838417 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bda54-21c9-4389-bdd5-4beebbda940c None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:31.838417 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bda54-21c9-4389-bdd5-4beebbda940c None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:31.839973 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 695/1369] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:18:31.845231 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6b370-d2fa-424f-88db-6b8306ffc20f None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:31.845528 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6b370-d2fa-424f-88db-6b8306ffc20f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:31.845810 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6b370-d2fa-424f-88db-6b8306ffc20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:31.879432 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:31.918245 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-93d6b370-d2fa-424f-88db-6b8306ffc20f None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:18:31.919485 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-93d6b370-d2fa-424f-88db-6b8306ffc20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:32.027479 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 675/1370] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:18:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:18:32.345420 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bccfa83-ae26-4c05-b3dc-b0bc7f1355df None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:32.345701 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bccfa83-ae26-4c05-b3dc-b0bc7f1355df None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:32.345986 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bccfa83-ae26-4c05-b3dc-b0bc7f1355df None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:32.347427 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 696/1371] 10.222.0.40 () {54 vars in 964 bytes} [Mon Jul 18 06:18:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:18:32.353221 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922e83c-3c07-4255-a724-472a6a5139e8 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:32.353740 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922e83c-3c07-4255-a724-472a6a5139e8 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:32.353863 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922e83c-3c07-4255-a724-472a6a5139e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:32.437017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-b922e83c-3c07-4255-a724-472a6a5139e8 None None] MFA Rules not processed for user `5af3a4a1bc1f4c4e9653d0a790f12606`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:18:32.438259 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b922e83c-3c07-4255-a724-472a6a5139e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:32.491524 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:32.525779 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 676/1372] 10.222.0.40 () {58 vars in 1061 bytes} [Mon Jul 18 06:18:32 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:18:32.533822 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bccfa83-ae26-4c05-b3dc-b0bc7f1355df None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:18:32.535832 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-2bccfa83-ae26-4c05-b3dc-b0bc7f1355df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:32.592947 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:32.648460 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bccfa83-ae26-4c05-b3dc-b0bc7f1355df None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:18:32.731541 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a228f104-2d76-444b-a705-44b2151c181c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:18:32.732202 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228f104-2d76-444b-a705-44b2151c181c service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:32.732528 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228f104-2d76-444b-a705-44b2151c181c service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:32.732813 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228f104-2d76-444b-a705-44b2151c181c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:32.734477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-a228f104-2d76-444b-a705-44b2151c181c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:32.841899 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:32.856759 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a228f104-2d76-444b-a705-44b2151c181c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:18:32.858500 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a228f104-2d76-444b-a705-44b2151c181c service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:18:32.878769 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 697/1373] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:18:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 06:18:33.399494 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d98c2-40d4-4e6c-bc0f-0858edb63a15 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:33.399960 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d98c2-40d4-4e6c-bc0f-0858edb63a15 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:33.400233 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d98c2-40d4-4e6c-bc0f-0858edb63a15 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:33.401802 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 677/1374] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:18:33.408064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d63987-0605-4e60-bef4-49aa2b858efd None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:33.408628 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d63987-0605-4e60-bef4-49aa2b858efd None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:33.409161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d63987-0605-4e60-bef4-49aa2b858efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:33.500248 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-93d63987-0605-4e60-bef4-49aa2b858efd None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:18:33.501644 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-93d63987-0605-4e60-bef4-49aa2b858efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:33.612103 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 698/1375] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:18:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:18:33.661455 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d98c2-40d4-4e6c-bc0f-0858edb63a15 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:18:33.663391 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-5f0d98c2-40d4-4e6c-bc0f-0858edb63a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:33.819068 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d98c2-40d4-4e6c-bc0f-0858edb63a15 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:18:33.874692 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:33.941185 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e4748-05d2-45a2-8811-c62c086fef5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:18:33.941838 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e4748-05d2-45a2-8811-c62c086fef5d service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:33.942118 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e4748-05d2-45a2-8811-c62c086fef5d service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:33.942936 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e4748-05d2-45a2-8811-c62c086fef5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:33.945009 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-313e4748-05d2-45a2-8811-c62c086fef5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:34.044709 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313e4748-05d2-45a2-8811-c62c086fef5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:18:34.046576 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313e4748-05d2-45a2-8811-c62c086fef5d service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:18:34.066216 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 678/1376] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:18:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:18:36.281142 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4661a-ac63-4544-8ac2-d554ea3a609c None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:36.281696 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4661a-ac63-4544-8ac2-d554ea3a609c None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:36.281696 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4661a-ac63-4544-8ac2-d554ea3a609c None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:36.283088 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 699/1377] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:18:36.289539 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ddcbe-369d-488a-87c2-7dba263bb681 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:36.290377 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ddcbe-369d-488a-87c2-7dba263bb681 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:36.290377 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ddcbe-369d-488a-87c2-7dba263bb681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:36.368806 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-e20ddcbe-369d-488a-87c2-7dba263bb681 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:18:36.370052 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e20ddcbe-369d-488a-87c2-7dba263bb681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:36.386776 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:36.490112 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 679/1378] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:18:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:18:36.552195 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f4661a-ac63-4544-8ac2-d554ea3a609c None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:18:36.554291 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-06f4661a-ac63-4544-8ac2-d554ea3a609c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:36.651441 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:36.775295 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f4661a-ac63-4544-8ac2-d554ea3a609c None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:18:36.902971 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb81f18-b8ef-4fe2-b71c-be86e6ce591d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:18:36.903638 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb81f18-b8ef-4fe2-b71c-be86e6ce591d service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:36.903936 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb81f18-b8ef-4fe2-b71c-be86e6ce591d service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:36.904218 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb81f18-b8ef-4fe2-b71c-be86e6ce591d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:36.905986 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-1fb81f18-b8ef-4fe2-b71c-be86e6ce591d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:36.927621 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:37.008728 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb81f18-b8ef-4fe2-b71c-be86e6ce591d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:18:37.010404 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb81f18-b8ef-4fe2-b71c-be86e6ce591d service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:18:37.031800 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 700/1379] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:18:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:18:39.330386 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2426ae4-34d4-47c0-9691-3b3c7b95672f None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:39.330995 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2426ae4-34d4-47c0-9691-3b3c7b95672f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:39.330995 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2426ae4-34d4-47c0-9691-3b3c7b95672f None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:39.332316 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 680/1380] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:18:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:18:39.338608 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff201302-92c1-46e4-b467-2e36627e5ca2 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:39.339122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff201302-92c1-46e4-b467-2e36627e5ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:39.339203 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff201302-92c1-46e4-b467-2e36627e5ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:39.421639 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-ff201302-92c1-46e4-b467-2e36627e5ca2 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:18:39.423314 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ff201302-92c1-46e4-b467-2e36627e5ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:39.536953 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:39.543627 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 701/1381] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:18:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:18:39.589255 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2426ae4-34d4-47c0-9691-3b3c7b95672f None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:18:39.590768 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f2426ae4-34d4-47c0-9691-3b3c7b95672f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:39.773771 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:39.786859 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2426ae4-34d4-47c0-9691-3b3c7b95672f None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:18:39.929236 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bde281-6535-471b-b0b4-1834ce56ae53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:18:39.929975 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bde281-6535-471b-b0b4-1834ce56ae53 service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:39.930291 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bde281-6535-471b-b0b4-1834ce56ae53 service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:39.930617 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bde281-6535-471b-b0b4-1834ce56ae53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:39.932355 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-56bde281-6535-471b-b0b4-1834ce56ae53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:40.043221 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bde281-6535-471b-b0b4-1834ce56ae53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:18:40.045032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bde281-6535-471b-b0b4-1834ce56ae53 service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:18:40.057158 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:40.066123 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 681/1382] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:18:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:18:42.651223 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e429351-700c-4ad0-8c89-c021dba5cb51 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:42.652093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e429351-700c-4ad0-8c89-c021dba5cb51 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:42.652266 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e429351-700c-4ad0-8c89-c021dba5cb51 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:42.654212 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 702/1383] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:18:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:18:42.662341 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73662966-3b89-416c-b499-3d11c3d63057 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:42.662998 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73662966-3b89-416c-b499-3d11c3d63057 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:42.662998 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73662966-3b89-416c-b499-3d11c3d63057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:42.749888 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-73662966-3b89-416c-b499-3d11c3d63057 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:18:42.751214 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-73662966-3b89-416c-b499-3d11c3d63057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:42.886327 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 682/1384] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:18:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:18:42.935436 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e429351-700c-4ad0-8c89-c021dba5cb51 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:18:42.937047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-7e429351-700c-4ad0-8c89-c021dba5cb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:43.106547 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e429351-700c-4ad0-8c89-c021dba5cb51 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:18:43.187860 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:43.261635 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d7f589-f267-41a5-b3d4-0f470fd41e13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:18:43.262320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7f589-f267-41a5-b3d4-0f470fd41e13 service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:43.262876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7f589-f267-41a5-b3d4-0f470fd41e13 service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:43.263150 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7f589-f267-41a5-b3d4-0f470fd41e13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:43.264902 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e4d7f589-f267-41a5-b3d4-0f470fd41e13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:43.365894 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d7f589-f267-41a5-b3d4-0f470fd41e13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:18:43.367579 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d7f589-f267-41a5-b3d4-0f470fd41e13 service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:18:43.389028 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 703/1385] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:18:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:18:45.525834 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e2af1-1f82-4df1-acc9-09887072c33c None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:45.526906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e2af1-1f82-4df1-acc9-09887072c33c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:45.526906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e2af1-1f82-4df1-acc9-09887072c33c None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:45.527792 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 683/1386] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:18:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:18:45.535226 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a061b-0fb2-4718-ba76-c21cb0059aca None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:45.536332 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a061b-0fb2-4718-ba76-c21cb0059aca None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:45.536332 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a061b-0fb2-4718-ba76-c21cb0059aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:45.642429 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-da9a061b-0fb2-4718-ba76-c21cb0059aca None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:18:45.643866 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-da9a061b-0fb2-4718-ba76-c21cb0059aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:45.687018 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:45.785459 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 704/1387] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:18:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:18:45.846526 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e2af1-1f82-4df1-acc9-09887072c33c None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:18:45.851347 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ba3e2af1-1f82-4df1-acc9-09887072c33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:45.953280 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:46.078771 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e2af1-1f82-4df1-acc9-09887072c33c None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:18:46.242297 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fee88e-87bb-4714-a775-0cfd7adca49f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:18:46.242998 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fee88e-87bb-4714-a775-0cfd7adca49f service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:46.243296 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fee88e-87bb-4714-a775-0cfd7adca49f service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:46.243581 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fee88e-87bb-4714-a775-0cfd7adca49f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:46.245507 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-87fee88e-87bb-4714-a775-0cfd7adca49f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:46.298006 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:46.373099 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fee88e-87bb-4714-a775-0cfd7adca49f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:18:46.374671 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fee88e-87bb-4714-a775-0cfd7adca49f service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:18:46.399219 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 684/1388] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:18:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:18:48.616998 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50688b-2555-48c6-b7f0-9addda255393 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:48.618453 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50688b-2555-48c6-b7f0-9addda255393 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:48.618453 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50688b-2555-48c6-b7f0-9addda255393 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:48.619652 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 705/1389] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:18:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:18:48.625961 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ee64e-135b-4a14-9dc8-1b8e1c3718be None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:48.626965 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ee64e-135b-4a14-9dc8-1b8e1c3718be None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:48.627488 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ee64e-135b-4a14-9dc8-1b8e1c3718be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:48.665821 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49d3f6-e324-4c45-ba48-2ecac4d9f7d1 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:48.666170 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49d3f6-e324-4c45-ba48-2ecac4d9f7d1 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:48.666389 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49d3f6-e324-4c45-ba48-2ecac4d9f7d1 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:48.667794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 706/1390] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:18:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:18:48.673518 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0ea95-6419-4657-99a6-e7e11d79ef61 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:48.682048 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0ea95-6419-4657-99a6-e7e11d79ef61 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:48.682048 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0ea95-6419-4657-99a6-e7e11d79ef61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:48.706007 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-653ee64e-135b-4a14-9dc8-1b8e1c3718be None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:18:48.707949 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-653ee64e-135b-4a14-9dc8-1b8e1c3718be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:48.746069 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-5cb0ea95-6419-4657-99a6-e7e11d79ef61 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:18:48.747310 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-5cb0ea95-6419-4657-99a6-e7e11d79ef61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:48.817488 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 685/1391] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:18:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:18:48.875331 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 707/1392] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:18:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:18:48.877269 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5686a-f2d3-4190-b849-d9cf4b20ef02 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:48.877734 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5686a-f2d3-4190-b849-d9cf4b20ef02 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:48.878186 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5686a-f2d3-4190-b849-d9cf4b20ef02 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:48.880331 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 686/1393] 10.222.0.40 () {54 vars in 961 bytes} [Mon Jul 18 06:18:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:18:48.890278 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c02b66-d373-490f-877a-59f647915caa None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:48.890583 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c02b66-d373-490f-877a-59f647915caa None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:48.890861 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c02b66-d373-490f-877a-59f647915caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:48.908487 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:48.922200 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c5686a-f2d3-4190-b849-d9cf4b20ef02 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:18:48.925352 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-14c5686a-f2d3-4190-b849-d9cf4b20ef02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:48.944714 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:48.974570 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-77c02b66-d373-490f-877a-59f647915caa None None] MFA Rules not processed for user `1193330cb5644765a7abb77738929ba4`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:18:48.975810 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-77c02b66-d373-490f-877a-59f647915caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:49.055314 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 708/1394] 10.222.0.40 () {58 vars in 1058 bytes} [Mon Jul 18 06:18:48 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:18:49.062942 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c02b66-d373-490f-877a-59f647915caa None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:18:49.072323 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c02b66-d373-490f-877a-59f647915caa None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:18:49.091507 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c5686a-f2d3-4190-b849-d9cf4b20ef02 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:18:49.153238 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a32ed0-8c69-42ec-b3e3-4d8298aaafb8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1193330cb5644765a7abb77738929ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:18:49.153897 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a32ed0-8c69-42ec-b3e3-4d8298aaafb8 service glance] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:49.154179 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a32ed0-8c69-42ec-b3e3-4d8298aaafb8 service glance] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:49.154466 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a32ed0-8c69-42ec-b3e3-4d8298aaafb8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:49.156100 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-f1a32ed0-8c69-42ec-b3e3-4d8298aaafb8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:49.184051 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:49.187839 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:49.213944 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd3f27-3aab-49da-acf1-a723c71262a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:18:49.214601 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd3f27-3aab-49da-acf1-a723c71262a7 service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:49.214886 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd3f27-3aab-49da-acf1-a723c71262a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:49.215164 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd3f27-3aab-49da-acf1-a723c71262a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:49.216966 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-43bd3f27-3aab-49da-acf1-a723c71262a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:49.293213 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a32ed0-8c69-42ec-b3e3-4d8298aaafb8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:18:49.294674 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a32ed0-8c69-42ec-b3e3-4d8298aaafb8 service glance] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:18:49.317544 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 709/1395] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:18:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 06:18:49.324595 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bd3f27-3aab-49da-acf1-a723c71262a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:18:49.326217 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bd3f27-3aab-49da-acf1-a723c71262a7 service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:18:49.344954 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 687/1396] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:18:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:18:50.222552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dbd2a8-ee11-4234-a9f3-bdc5bb8eb7d5 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:50.223585 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dbd2a8-ee11-4234-a9f3-bdc5bb8eb7d5 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:50.223933 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dbd2a8-ee11-4234-a9f3-bdc5bb8eb7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:50.302087 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-88dbd2a8-ee11-4234-a9f3-bdc5bb8eb7d5 None None] MFA Rules not processed for user `b84dde57ff8e47419866377a819bea21`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:18:50.303314 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-88dbd2a8-ee11-4234-a9f3-bdc5bb8eb7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:50.387998 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:50.394242 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 710/1397] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Jul 18 06:18:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:18:50.439428 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f93491-596b-4572-9891-76969348bf40 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:50.440141 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f93491-596b-4572-9891-76969348bf40 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:50.440141 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f93491-596b-4572-9891-76969348bf40 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:50.441879 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 688/1398] 10.222.0.40 () {54 vars in 943 bytes} [Mon Jul 18 06:18:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:18:50.448822 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450aca81-5f9b-4ee4-a953-30c256d00074 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:50.449124 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450aca81-5f9b-4ee4-a953-30c256d00074 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:50.449411 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450aca81-5f9b-4ee4-a953-30c256d00074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:50.532103 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-450aca81-5f9b-4ee4-a953-30c256d00074 None None] MFA Rules not processed for user `0681f5fbea6d415c9ad2f3afae256a5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:18:50.533268 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-450aca81-5f9b-4ee4-a953-30c256d00074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:50.649194 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 711/1399] 10.222.0.40 () {58 vars in 1040 bytes} [Mon Jul 18 06:18:50 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:18:50.658671 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f93491-596b-4572-9891-76969348bf40 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:18:50.660927 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-72f93491-596b-4572-9891-76969348bf40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:50.766079 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:50.808841 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f93491-596b-4572-9891-76969348bf40 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:18:50.918096 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed29ba0-1ef0-4ec3-9985-df04fcb915b3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0681f5fbea6d415c9ad2f3afae256a5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:18:50.918751 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed29ba0-1ef0-4ec3-9985-df04fcb915b3 service swift] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:50.919036 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed29ba0-1ef0-4ec3-9985-df04fcb915b3 service swift] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:50.919317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed29ba0-1ef0-4ec3-9985-df04fcb915b3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:50.921015 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-fed29ba0-1ef0-4ec3-9985-df04fcb915b3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:51.010451 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed29ba0-1ef0-4ec3-9985-df04fcb915b3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:18:51.012072 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed29ba0-1ef0-4ec3-9985-df04fcb915b3 service swift] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:18:51.016141 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:51.022361 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 689/1400] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Jul 18 06:18:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 06:18:51.555451 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f699902-30b7-4fcd-81d7-e0a10aee898f None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:51.555967 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f699902-30b7-4fcd-81d7-e0a10aee898f None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:51.555967 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f699902-30b7-4fcd-81d7-e0a10aee898f None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:51.557501 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 712/1401] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:18:51.562795 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea3b7f-794d-478c-a906-d0493b328558 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:51.563240 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea3b7f-794d-478c-a906-d0493b328558 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:51.563382 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea3b7f-794d-478c-a906-d0493b328558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:51.634118 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-14ea3b7f-794d-478c-a906-d0493b328558 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:18:51.635485 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-14ea3b7f-794d-478c-a906-d0493b328558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:51.761272 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 690/1402] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:18:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:18:51.809046 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f699902-30b7-4fcd-81d7-e0a10aee898f None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:18:51.810858 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4f699902-30b7-4fcd-81d7-e0a10aee898f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:51.879898 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:51.989470 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f699902-30b7-4fcd-81d7-e0a10aee898f None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:18:52.123532 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de58ee-19fa-41ad-be30-c7b509747e60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:18:52.124113 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de58ee-19fa-41ad-be30-c7b509747e60 service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:52.124563 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de58ee-19fa-41ad-be30-c7b509747e60 service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:52.124853 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de58ee-19fa-41ad-be30-c7b509747e60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:52.126552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-a6de58ee-19fa-41ad-be30-c7b509747e60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:52.147600 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:52.236048 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6de58ee-19fa-41ad-be30-c7b509747e60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:18:52.237726 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6de58ee-19fa-41ad-be30-c7b509747e60 service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:18:52.260367 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 713/1403] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:18:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:18:54.505189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3723161b-03dc-4ab8-a540-39ef7ae34c20 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:54.506088 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3723161b-03dc-4ab8-a540-39ef7ae34c20 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:54.506538 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3723161b-03dc-4ab8-a540-39ef7ae34c20 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:54.508108 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 691/1404] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:18:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:18:54.513614 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427504fd-3b6d-4738-b91b-1dcda87666c9 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:54.514082 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427504fd-3b6d-4738-b91b-1dcda87666c9 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:54.514368 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427504fd-3b6d-4738-b91b-1dcda87666c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:54.591704 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-427504fd-3b6d-4738-b91b-1dcda87666c9 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:18:54.592967 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-427504fd-3b6d-4738-b91b-1dcda87666c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:54.702578 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:54.712468 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 714/1405] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:18:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:18:54.757893 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3723161b-03dc-4ab8-a540-39ef7ae34c20 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:18:54.759434 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-3723161b-03dc-4ab8-a540-39ef7ae34c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:54.848080 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:54.922169 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3723161b-03dc-4ab8-a540-39ef7ae34c20 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:18:55.052021 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a464d4a4-7b79-4e4d-b3d8-73e48a9114df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:18:55.053029 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464d4a4-7b79-4e4d-b3d8-73e48a9114df service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:55.053502 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464d4a4-7b79-4e4d-b3d8-73e48a9114df service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:55.053791 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464d4a4-7b79-4e4d-b3d8-73e48a9114df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:55.055533 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a464d4a4-7b79-4e4d-b3d8-73e48a9114df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:55.105130 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:55.163136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a464d4a4-7b79-4e4d-b3d8-73e48a9114df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:18:55.164785 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a464d4a4-7b79-4e4d-b3d8-73e48a9114df service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:18:55.192662 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 692/1406] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:18:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:18:57.416654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1d7e3-bfdf-497b-b0af-31e6c1fec756 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:57.417217 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1d7e3-bfdf-497b-b0af-31e6c1fec756 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:57.417217 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1d7e3-bfdf-497b-b0af-31e6c1fec756 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:57.418599 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 715/1407] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:18:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:18:57.423819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24356f1c-2b35-42f8-b021-9f6fc7ee44f8 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:57.424244 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24356f1c-2b35-42f8-b021-9f6fc7ee44f8 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:57.424468 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24356f1c-2b35-42f8-b021-9f6fc7ee44f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:57.498928 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-24356f1c-2b35-42f8-b021-9f6fc7ee44f8 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:18:57.500105 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-24356f1c-2b35-42f8-b021-9f6fc7ee44f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:57.629931 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 693/1408] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:18:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:18:57.675057 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1d7e3-bfdf-497b-b0af-31e6c1fec756 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:18:57.676757 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c1e1d7e3-bfdf-497b-b0af-31e6c1fec756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:57.850384 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1d7e3-bfdf-497b-b0af-31e6c1fec756 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:18:57.929091 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:18:57.982901 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d9a4d-22b5-4685-8de4-9dcc67043c37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:18:57.983559 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d9a4d-22b5-4685-8de4-9dcc67043c37 service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:18:57.983847 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d9a4d-22b5-4685-8de4-9dcc67043c37 service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:18:57.984125 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d9a4d-22b5-4685-8de4-9dcc67043c37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:18:57.985820 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-055d9a4d-22b5-4685-8de4-9dcc67043c37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:18:58.088887 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055d9a4d-22b5-4685-8de4-9dcc67043c37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:18:58.090608 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055d9a4d-22b5-4685-8de4-9dcc67043c37 service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:18:58.109603 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 716/1409] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:18:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:19:07.625143 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=93 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:08.107048 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=65 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:15.102492 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d80cf-3b97-4c35-9455-d1d80e3dfb97 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:15.103012 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d80cf-3b97-4c35-9455-d1d80e3dfb97 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:15.103012 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d80cf-3b97-4c35-9455-d1d80e3dfb97 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:15.104345 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 694/1410] 10.222.0.40 () {54 vars in 918 bytes} [Mon Jul 18 06:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:19:15.109693 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99debe07-19a1-4709-bb02-45458a48075c None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:15.110115 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99debe07-19a1-4709-bb02-45458a48075c None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:15.110260 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99debe07-19a1-4709-bb02-45458a48075c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:15.187635 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-99debe07-19a1-4709-bb02-45458a48075c None None] MFA Rules not processed for user `a3560534427544cdabfd0ba702a0a165`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:19:15.188889 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-99debe07-19a1-4709-bb02-45458a48075c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:15.297520 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 717/1411] 10.222.0.40 () {58 vars in 1015 bytes} [Mon Jul 18 06:19:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:19:15.313037 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10617d6-fca4-4029-8a23-cdac342a8051 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:15.313336 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10617d6-fca4-4029-8a23-cdac342a8051 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:15.313620 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10617d6-fca4-4029-8a23-cdac342a8051 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:15.314987 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 695/1412] 10.222.0.40 () {54 vars in 918 bytes} [Mon Jul 18 06:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:19:15.320468 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07d8d5-ef3c-4783-9bf3-34a4f25a614b None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:15.320765 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07d8d5-ef3c-4783-9bf3-34a4f25a614b None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:15.321048 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07d8d5-ef3c-4783-9bf3-34a4f25a614b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:15.393988 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:15.397104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-db07d8d5-ef3c-4783-9bf3-34a4f25a614b None None] MFA Rules not processed for user `f92d948810ea4241810473506b654f7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:19:15.398332 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-db07d8d5-ef3c-4783-9bf3-34a4f25a614b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:15.514189 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 718/1413] 10.222.0.40 () {58 vars in 1015 bytes} [Mon Jul 18 06:19:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:19:17.982918 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd8deb-0308-4cf1-a505-b738b2ad92c6 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:17.983525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd8deb-0308-4cf1-a505-b738b2ad92c6 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:17.983525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd8deb-0308-4cf1-a505-b738b2ad92c6 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:17.984798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 696/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 06:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:19:17.990532 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f3ab5-1071-4231-aff2-9d030e6210be None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:17.991008 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f3ab5-1071-4231-aff2-9d030e6210be None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:17.991111 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f3ab5-1071-4231-aff2-9d030e6210be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:18.056948 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-677f3ab5-1071-4231-aff2-9d030e6210be None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:19:18.058156 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-677f3ab5-1071-4231-aff2-9d030e6210be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:18.114682 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:18.157050 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 719/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 06:19:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:19:18.199400 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd8deb-0308-4cf1-a505-b738b2ad92c6 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:19:18.200994 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-81cd8deb-0308-4cf1-a505-b738b2ad92c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:18.348075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd8deb-0308-4cf1-a505-b738b2ad92c6 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:19:18.419739 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:18.461083 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f56e2a-1834-4cf8-ba53-62ef30af230b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:19:18.461733 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f56e2a-1834-4cf8-ba53-62ef30af230b service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:18.462018 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f56e2a-1834-4cf8-ba53-62ef30af230b service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:18.462295 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f56e2a-1834-4cf8-ba53-62ef30af230b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:18.463949 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f7f56e2a-1834-4cf8-ba53-62ef30af230b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:18.556902 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f56e2a-1834-4cf8-ba53-62ef30af230b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:19:18.558492 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f56e2a-1834-4cf8-ba53-62ef30af230b service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:19:18.577560 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 697/1416] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:19:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:19:19.479388 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009952a-4b18-4082-a08c-b8f352fda631 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:19.479916 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009952a-4b18-4082-a08c-b8f352fda631 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:19.480021 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009952a-4b18-4082-a08c-b8f352fda631 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:19.481428 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 720/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 06:19:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:19:19.486403 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa33c8e-7cb7-4cdc-a6cd-91b3e0bf1947 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:19.486837 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa33c8e-7cb7-4cdc-a6cd-91b3e0bf1947 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:19.486964 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa33c8e-7cb7-4cdc-a6cd-91b3e0bf1947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:19.564191 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-ffa33c8e-7cb7-4cdc-a6cd-91b3e0bf1947 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:19:19.565389 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ffa33c8e-7cb7-4cdc-a6cd-91b3e0bf1947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:19.592740 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:19.663684 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 698/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 06:19:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:19:19.816210 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009952a-4b18-4082-a08c-b8f352fda631 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:19:19.817674 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d009952a-4b18-4082-a08c-b8f352fda631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:19.963808 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009952a-4b18-4082-a08c-b8f352fda631 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:19:20.005131 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:20.078948 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc77eb-55ae-4337-af2c-059d8cfe46b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:19:20.079529 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc77eb-55ae-4337-af2c-059d8cfe46b4 service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:20.079816 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc77eb-55ae-4337-af2c-059d8cfe46b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:20.080093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc77eb-55ae-4337-af2c-059d8cfe46b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:20.081805 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-05fc77eb-55ae-4337-af2c-059d8cfe46b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:20.171324 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc77eb-55ae-4337-af2c-059d8cfe46b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:19:20.172947 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc77eb-55ae-4337-af2c-059d8cfe46b4 service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:19:20.191348 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 721/1419] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:19:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:19:21.085201 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c95f9-8c4c-4700-81a5-d0bbce877cd2 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:21.085807 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c95f9-8c4c-4700-81a5-d0bbce877cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:21.085807 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c95f9-8c4c-4700-81a5-d0bbce877cd2 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:21.087142 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 699/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 06:19:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:19:21.092494 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147d32f-68ff-418e-9b3f-f589c6dfd57e None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:21.092980 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147d32f-68ff-418e-9b3f-f589c6dfd57e None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:21.093081 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147d32f-68ff-418e-9b3f-f589c6dfd57e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:21.147274 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:21.169683 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-5147d32f-68ff-418e-9b3f-f589c6dfd57e None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:19:21.170956 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-5147d32f-68ff-418e-9b3f-f589c6dfd57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:21.271908 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 722/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 06:19:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:19:21.423777 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c95f9-8c4c-4700-81a5-d0bbce877cd2 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:19:21.425338 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-591c95f9-8c4c-4700-81a5-d0bbce877cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:21.578483 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c95f9-8c4c-4700-81a5-d0bbce877cd2 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:19:21.591049 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:21.693975 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6b6cd-efda-41c0-9b8e-cbfd5ab91fde service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:19:21.694631 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6b6cd-efda-41c0-9b8e-cbfd5ab91fde service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:21.695227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6b6cd-efda-41c0-9b8e-cbfd5ab91fde service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:21.695227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6b6cd-efda-41c0-9b8e-cbfd5ab91fde service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:21.697172 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-0cb6b6cd-efda-41c0-9b8e-cbfd5ab91fde service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:21.789507 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb6b6cd-efda-41c0-9b8e-cbfd5ab91fde service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:19:21.791119 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb6b6cd-efda-41c0-9b8e-cbfd5ab91fde service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:19:21.809727 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 700/1422] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:19:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:19:22.679767 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235e3f6-87c6-4e12-9663-a1656309098e None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:22.680297 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235e3f6-87c6-4e12-9663-a1656309098e None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:22.680297 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235e3f6-87c6-4e12-9663-a1656309098e None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:22.685599 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 723/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 06:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:19:22.687286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef9b26-4bd9-4014-9bcc-e982e6aed96d None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:22.687666 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef9b26-4bd9-4014-9bcc-e982e6aed96d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:22.687896 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef9b26-4bd9-4014-9bcc-e982e6aed96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:22.707239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:22.765993 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-e9ef9b26-4bd9-4014-9bcc-e982e6aed96d None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:19:22.767213 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e9ef9b26-4bd9-4014-9bcc-e982e6aed96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:22.864979 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 701/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 06:19:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:19:23.015709 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235e3f6-87c6-4e12-9663-a1656309098e None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:19:23.017194 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c235e3f6-87c6-4e12-9663-a1656309098e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:23.128886 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:23.168002 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235e3f6-87c6-4e12-9663-a1656309098e None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:19:23.284737 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e6f45e-eaa8-413f-b0d6-da8c0a8afde5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:19:23.285316 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6f45e-eaa8-413f-b0d6-da8c0a8afde5 service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:23.285605 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6f45e-eaa8-413f-b0d6-da8c0a8afde5 service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:23.286482 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6f45e-eaa8-413f-b0d6-da8c0a8afde5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:23.288244 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-39e6f45e-eaa8-413f-b0d6-da8c0a8afde5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:23.366605 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:23.380651 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e6f45e-eaa8-413f-b0d6-da8c0a8afde5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:19:23.385108 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e6f45e-eaa8-413f-b0d6-da8c0a8afde5 service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:19:23.402560 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 724/1425] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:19:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:19:24.280476 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdaa473-324c-45b1-b553-bbec6157dbb6 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:24.281047 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdaa473-324c-45b1-b553-bbec6157dbb6 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:24.281047 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdaa473-324c-45b1-b553-bbec6157dbb6 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:24.282464 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 702/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 06:19:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:19:24.287885 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf28ab-a9c5-4995-ad40-8e17e5b45d89 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:24.288370 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf28ab-a9c5-4995-ad40-8e17e5b45d89 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:24.288501 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf28ab-a9c5-4995-ad40-8e17e5b45d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:24.367632 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-8aaf28ab-a9c5-4995-ad40-8e17e5b45d89 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:19:24.368916 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-8aaf28ab-a9c5-4995-ad40-8e17e5b45d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:24.470486 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 725/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 06:19:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:19:24.623126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdaa473-324c-45b1-b553-bbec6157dbb6 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:19:24.624675 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-2cdaa473-324c-45b1-b553-bbec6157dbb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:24.711209 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:24.779031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdaa473-324c-45b1-b553-bbec6157dbb6 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:19:24.904849 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f9c89-e36c-4867-a768-1def4d357b43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:19:24.905508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f9c89-e36c-4867-a768-1def4d357b43 service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:24.905795 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f9c89-e36c-4867-a768-1def4d357b43 service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:24.906078 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f9c89-e36c-4867-a768-1def4d357b43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:24.907740 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-904f9c89-e36c-4867-a768-1def4d357b43 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:24.958390 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:25.000274 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f9c89-e36c-4867-a768-1def4d357b43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:19:25.001982 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f9c89-e36c-4867-a768-1def4d357b43 service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:19:25.021921 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 703/1428] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:19:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:19:25.905843 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e03a4-3d98-453d-9d9c-67f0f1d5f672 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:25.906417 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e03a4-3d98-453d-9d9c-67f0f1d5f672 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:25.906417 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e03a4-3d98-453d-9d9c-67f0f1d5f672 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:25.907774 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 726/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 06:19:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:19:25.913441 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ce91e-6d17-48fd-85c9-6bc7d5fd847a None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:25.913910 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ce91e-6d17-48fd-85c9-6bc7d5fd847a None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:25.914029 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ce91e-6d17-48fd-85c9-6bc7d5fd847a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:25.988660 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-906ce91e-6d17-48fd-85c9-6bc7d5fd847a None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:19:25.990092 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-906ce91e-6d17-48fd-85c9-6bc7d5fd847a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:26.090318 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 704/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 06:19:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:19:26.242384 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e03a4-3d98-453d-9d9c-67f0f1d5f672 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:19:26.243833 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-3c6e03a4-3d98-453d-9d9c-67f0f1d5f672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:26.295362 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:26.393565 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e03a4-3d98-453d-9d9c-67f0f1d5f672 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:19:26.510805 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f1b0a-51d5-45b6-a790-d6b74607ba45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:19:26.511630 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f1b0a-51d5-45b6-a790-d6b74607ba45 service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:26.511915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f1b0a-51d5-45b6-a790-d6b74607ba45 service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:26.512186 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f1b0a-51d5-45b6-a790-d6b74607ba45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:26.513872 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-819f1b0a-51d5-45b6-a790-d6b74607ba45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:26.521785 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:26.611736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819f1b0a-51d5-45b6-a790-d6b74607ba45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:19:26.613398 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819f1b0a-51d5-45b6-a790-d6b74607ba45 service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:19:26.632888 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 727/1431] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:19:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:19:27.517142 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ed770-d774-4053-953d-08fd2d721e13 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:27.517568 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ed770-d774-4053-953d-08fd2d721e13 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:27.517798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ed770-d774-4053-953d-08fd2d721e13 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:27.519196 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 705/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 06:19:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:19:27.525456 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611d5de-2e2e-4211-9d4a-ad79a825be92 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:27.525993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611d5de-2e2e-4211-9d4a-ad79a825be92 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:27.525993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611d5de-2e2e-4211-9d4a-ad79a825be92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:27.613950 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-d611d5de-2e2e-4211-9d4a-ad79a825be92 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:19:27.615049 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d611d5de-2e2e-4211-9d4a-ad79a825be92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:27.687361 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:27.718611 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 728/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 06:19:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:19:27.872308 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03ed770-d774-4053-953d-08fd2d721e13 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:19:27.873782 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a03ed770-d774-4053-953d-08fd2d721e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:27.892197 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:28.022352 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03ed770-d774-4053-953d-08fd2d721e13 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:19:28.110854 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:28.139115 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e9d633-bc7f-41f7-8d03-5ca4db4449d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:19:28.139766 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9d633-bc7f-41f7-8d03-5ca4db4449d4 service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:28.140050 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9d633-bc7f-41f7-8d03-5ca4db4449d4 service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:28.140326 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9d633-bc7f-41f7-8d03-5ca4db4449d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:28.142039 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b9e9d633-bc7f-41f7-8d03-5ca4db4449d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:28.233596 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e9d633-bc7f-41f7-8d03-5ca4db4449d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:19:28.235207 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e9d633-bc7f-41f7-8d03-5ca4db4449d4 service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:19:28.255180 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 706/1434] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:19:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:19:29.128648 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e927edea-016e-416b-a7ba-f16aac7a100b None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:29.129717 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e927edea-016e-416b-a7ba-f16aac7a100b None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:29.129717 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e927edea-016e-416b-a7ba-f16aac7a100b None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:29.131204 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 729/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 06:19:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:19:29.136498 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdbecfe-d5ba-4c1b-bb2f-d2b5eeb7cac6 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:29.136932 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdbecfe-d5ba-4c1b-bb2f-d2b5eeb7cac6 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:29.137066 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdbecfe-d5ba-4c1b-bb2f-d2b5eeb7cac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:29.213517 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-cbdbecfe-d5ba-4c1b-bb2f-d2b5eeb7cac6 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:19:29.214758 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-cbdbecfe-d5ba-4c1b-bb2f-d2b5eeb7cac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:29.257088 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:29.315303 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 707/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 06:19:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:19:29.471068 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e927edea-016e-416b-a7ba-f16aac7a100b None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:19:29.472552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e927edea-016e-416b-a7ba-f16aac7a100b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:29.627223 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e927edea-016e-416b-a7ba-f16aac7a100b None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:19:29.684576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:29.747675 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ee757-0aca-4b6d-a6ee-2f51e2eb4c03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:19:29.748361 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ee757-0aca-4b6d-a6ee-2f51e2eb4c03 service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:29.748696 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ee757-0aca-4b6d-a6ee-2f51e2eb4c03 service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:29.748981 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ee757-0aca-4b6d-a6ee-2f51e2eb4c03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:29.750764 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-764ee757-0aca-4b6d-a6ee-2f51e2eb4c03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:29.851854 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764ee757-0aca-4b6d-a6ee-2f51e2eb4c03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:19:29.853646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764ee757-0aca-4b6d-a6ee-2f51e2eb4c03 service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:19:29.874499 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 730/1437] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:19:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:19:30.822724 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7d75b-272d-4640-aa1b-a3f61fa845c5 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:30.823396 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7d75b-272d-4640-aa1b-a3f61fa845c5 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:30.823396 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7d75b-272d-4640-aa1b-a3f61fa845c5 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:30.824761 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 708/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 06:19:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:19:30.829841 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d339c6-eb61-4936-a1ee-3bb36f857ed9 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:30.830308 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d339c6-eb61-4936-a1ee-3bb36f857ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:30.830420 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d339c6-eb61-4936-a1ee-3bb36f857ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:30.909277 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-08d339c6-eb61-4936-a1ee-3bb36f857ed9 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:19:30.910509 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-08d339c6-eb61-4936-a1ee-3bb36f857ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:30.912688 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:31.007921 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 731/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 06:19:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:19:31.157603 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf7d75b-272d-4640-aa1b-a3f61fa845c5 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:19:31.159130 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-0bf7d75b-272d-4640-aa1b-a3f61fa845c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:31.309742 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf7d75b-272d-4640-aa1b-a3f61fa845c5 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:19:31.337707 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:31.425518 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac55b6-be03-49dc-9792-573c1e9f2b15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:19:31.426764 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac55b6-be03-49dc-9792-573c1e9f2b15 service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:31.427071 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac55b6-be03-49dc-9792-573c1e9f2b15 service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:31.427337 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac55b6-be03-49dc-9792-573c1e9f2b15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:31.429042 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e1ac55b6-be03-49dc-9792-573c1e9f2b15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:31.522052 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac55b6-be03-49dc-9792-573c1e9f2b15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:19:31.523567 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac55b6-be03-49dc-9792-573c1e9f2b15 service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:19:31.542390 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 709/1440] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:19:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:19:33.462005 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f55010-b012-4444-8d70-ac355b67d7f3 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:33.462543 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f55010-b012-4444-8d70-ac355b67d7f3 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:33.462664 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f55010-b012-4444-8d70-ac355b67d7f3 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:33.464093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 732/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 06:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:19:33.469598 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbcc61-7b32-4327-b7e3-5e98dab13778 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:33.470034 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbcc61-7b32-4327-b7e3-5e98dab13778 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:33.470161 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbcc61-7b32-4327-b7e3-5e98dab13778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:33.513087 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:33.538758 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-58bbcc61-7b32-4327-b7e3-5e98dab13778 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:19:33.540056 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-58bbcc61-7b32-4327-b7e3-5e98dab13778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:33.642038 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 710/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 06:19:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:19:33.682265 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f55010-b012-4444-8d70-ac355b67d7f3 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:19:33.685951 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-98f55010-b012-4444-8d70-ac355b67d7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:33.790729 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f55010-b012-4444-8d70-ac355b67d7f3 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:19:33.824054 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:33.867499 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3551a4-4791-4ed0-a49d-88993213593f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:19:33.868164 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3551a4-4791-4ed0-a49d-88993213593f service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:33.868491 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3551a4-4791-4ed0-a49d-88993213593f service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:33.868783 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3551a4-4791-4ed0-a49d-88993213593f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:33.870445 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ec3551a4-4791-4ed0-a49d-88993213593f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:33.984509 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3551a4-4791-4ed0-a49d-88993213593f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:19:33.985907 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3551a4-4791-4ed0-a49d-88993213593f service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:19:34.005233 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 733/1443] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:19:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:19:36.176549 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4534d32-3834-4f77-b31d-3f1690b9ea18 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:36.177357 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4534d32-3834-4f77-b31d-3f1690b9ea18 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:36.177357 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4534d32-3834-4f77-b31d-3f1690b9ea18 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:36.178562 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 711/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 06:19:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:19:36.183977 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53d864-ccbf-4ceb-a4f6-052ed818b8f7 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:36.184458 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53d864-ccbf-4ceb-a4f6-052ed818b8f7 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:36.184578 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53d864-ccbf-4ceb-a4f6-052ed818b8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:36.250049 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-1f53d864-ccbf-4ceb-a4f6-052ed818b8f7 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:19:36.251298 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-1f53d864-ccbf-4ceb-a4f6-052ed818b8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:36.265499 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:36.361813 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 734/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 06:19:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:19:36.401399 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4534d32-3834-4f77-b31d-3f1690b9ea18 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:19:36.402940 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f4534d32-3834-4f77-b31d-3f1690b9ea18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:36.509184 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4534d32-3834-4f77-b31d-3f1690b9ea18 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:19:36.521850 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:36.585204 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b160eb-3d18-4746-a1be-5cd2506c0c88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:19:36.585861 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b160eb-3d18-4746-a1be-5cd2506c0c88 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:36.586147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b160eb-3d18-4746-a1be-5cd2506c0c88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:36.586429 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b160eb-3d18-4746-a1be-5cd2506c0c88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:36.588158 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a8b160eb-3d18-4746-a1be-5cd2506c0c88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:36.703304 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b160eb-3d18-4746-a1be-5cd2506c0c88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:19:36.704801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b160eb-3d18-4746-a1be-5cd2506c0c88 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:19:36.724102 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 712/1446] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:19:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:19:39.145143 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3887d-b45c-4e64-92f6-d8f54c3e087f None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:39.145705 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3887d-b45c-4e64-92f6-d8f54c3e087f None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:39.145705 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3887d-b45c-4e64-92f6-d8f54c3e087f None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:39.147075 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 735/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 06:19:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:19:39.152672 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86ffd5-9c13-4fdf-bd61-d05e9bf8cc15 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:39.153133 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86ffd5-9c13-4fdf-bd61-d05e9bf8cc15 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:39.153253 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86ffd5-9c13-4fdf-bd61-d05e9bf8cc15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:39.201549 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:39.219962 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-9e86ffd5-9c13-4fdf-bd61-d05e9bf8cc15 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:19:39.221266 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-9e86ffd5-9c13-4fdf-bd61-d05e9bf8cc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:39.322983 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 713/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 06:19:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:19:39.361750 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee3887d-b45c-4e64-92f6-d8f54c3e087f None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:19:39.364062 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-aee3887d-b45c-4e64-92f6-d8f54c3e087f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:39.468232 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee3887d-b45c-4e64-92f6-d8f54c3e087f None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:19:39.509196 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:39.543358 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eff274b-a3a3-4e26-93ec-5faa6cc94209 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:19:39.544014 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff274b-a3a3-4e26-93ec-5faa6cc94209 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:39.544299 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff274b-a3a3-4e26-93ec-5faa6cc94209 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:39.544618 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff274b-a3a3-4e26-93ec-5faa6cc94209 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:39.546280 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-0eff274b-a3a3-4e26-93ec-5faa6cc94209 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:39.658240 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eff274b-a3a3-4e26-93ec-5faa6cc94209 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:19:39.659690 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eff274b-a3a3-4e26-93ec-5faa6cc94209 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:19:39.679905 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 736/1449] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:19:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:19:41.888375 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a0eb4-3d33-434d-924d-3e88bcf5c3f5 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:41.888933 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a0eb4-3d33-434d-924d-3e88bcf5c3f5 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:41.888933 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a0eb4-3d33-434d-924d-3e88bcf5c3f5 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:41.890322 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 714/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 06:19:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:19:41.895760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b9d01-0521-4d1d-94a3-78c1979b8b61 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:41.896186 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b9d01-0521-4d1d-94a3-78c1979b8b61 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:41.896366 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b9d01-0521-4d1d-94a3-78c1979b8b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:41.961234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-9b7b9d01-0521-4d1d-94a3-78c1979b8b61 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:19:41.962401 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-9b7b9d01-0521-4d1d-94a3-78c1979b8b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:41.983526 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:42.063327 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 737/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 06:19:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:19:42.102046 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a0eb4-3d33-434d-924d-3e88bcf5c3f5 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:19:42.103586 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-769a0eb4-3d33-434d-924d-3e88bcf5c3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:42.209610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a0eb4-3d33-434d-924d-3e88bcf5c3f5 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:19:42.230703 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:42.294200 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6269005-3c22-4d9f-a26e-cd49c7e51c8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:19:42.295000 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6269005-3c22-4d9f-a26e-cd49c7e51c8a service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:42.295359 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6269005-3c22-4d9f-a26e-cd49c7e51c8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:42.295637 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6269005-3c22-4d9f-a26e-cd49c7e51c8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:42.297350 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-c6269005-3c22-4d9f-a26e-cd49c7e51c8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:42.406472 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6269005-3c22-4d9f-a26e-cd49c7e51c8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:19:42.407894 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6269005-3c22-4d9f-a26e-cd49c7e51c8a service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:19:42.426668 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 715/1452] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:19:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:19:44.544496 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504173b-dfde-4881-a485-6c56624191c2 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:44.545044 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504173b-dfde-4881-a485-6c56624191c2 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:44.545044 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504173b-dfde-4881-a485-6c56624191c2 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:44.546474 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 738/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 06:19:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:19:44.552281 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a9e2a-af11-4749-abc7-d955fefdbe5b None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:44.552859 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a9e2a-af11-4749-abc7-d955fefdbe5b None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:44.552956 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a9e2a-af11-4749-abc7-d955fefdbe5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:44.620370 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:44.623325 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-f45a9e2a-af11-4749-abc7-d955fefdbe5b None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:19:44.624619 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f45a9e2a-af11-4749-abc7-d955fefdbe5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:44.728477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 716/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 06:19:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:19:44.766595 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a504173b-dfde-4881-a485-6c56624191c2 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:19:44.768146 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-a504173b-dfde-4881-a485-6c56624191c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:44.873793 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a504173b-dfde-4881-a485-6c56624191c2 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:19:44.924908 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:44.950909 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b652bd-1cef-4fc2-a188-45a1413bf004 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:19:44.951565 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b652bd-1cef-4fc2-a188-45a1413bf004 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:44.951851 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b652bd-1cef-4fc2-a188-45a1413bf004 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:44.952168 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b652bd-1cef-4fc2-a188-45a1413bf004 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:44.954605 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-53b652bd-1cef-4fc2-a188-45a1413bf004 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:45.069136 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b652bd-1cef-4fc2-a188-45a1413bf004 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:19:45.070640 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b652bd-1cef-4fc2-a188-45a1413bf004 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:19:45.090351 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 739/1455] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:19:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:19:47.173842 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20be519-a8a2-409b-b19f-c70be84308e0 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:47.174422 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20be519-a8a2-409b-b19f-c70be84308e0 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:47.174422 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20be519-a8a2-409b-b19f-c70be84308e0 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:47.176431 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 717/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 06:19:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:19:47.181944 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1091255-685c-4c20-927f-70a96f8250ef None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:47.182376 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1091255-685c-4c20-927f-70a96f8250ef None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:47.182493 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1091255-685c-4c20-927f-70a96f8250ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:47.247843 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-c1091255-685c-4c20-927f-70a96f8250ef None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:19:47.249082 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c1091255-685c-4c20-927f-70a96f8250ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:47.280277 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:47.349846 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 740/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 06:19:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:19:47.389180 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20be519-a8a2-409b-b19f-c70be84308e0 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:19:47.390750 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f20be519-a8a2-409b-b19f-c70be84308e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:47.495162 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20be519-a8a2-409b-b19f-c70be84308e0 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:19:47.524884 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:47.571412 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb3283-c19a-4d56-a8d7-c031664713a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:19:47.572077 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb3283-c19a-4d56-a8d7-c031664713a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:47.572355 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb3283-c19a-4d56-a8d7-c031664713a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:47.572672 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb3283-c19a-4d56-a8d7-c031664713a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:47.574327 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-76eb3283-c19a-4d56-a8d7-c031664713a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:47.690952 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eb3283-c19a-4d56-a8d7-c031664713a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:19:47.690952 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eb3283-c19a-4d56-a8d7-c031664713a2 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:19:47.708262 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 718/1458] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:19:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:19:49.782857 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3125e5-d7f6-4b54-8982-2d885e77414d None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:49.783395 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3125e5-d7f6-4b54-8982-2d885e77414d None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:49.783395 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3125e5-d7f6-4b54-8982-2d885e77414d None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:49.784874 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 741/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 06:19:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:19:49.790402 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31b38e-7017-4d6b-bec8-2ac7d4831e3e None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:49.790883 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31b38e-7017-4d6b-bec8-2ac7d4831e3e None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:49.791002 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31b38e-7017-4d6b-bec8-2ac7d4831e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:49.860492 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-ce31b38e-7017-4d6b-bec8-2ac7d4831e3e None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:19:49.861723 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ce31b38e-7017-4d6b-bec8-2ac7d4831e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:49.870940 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:49.965935 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 719/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 06:19:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:19:50.004997 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3125e5-d7f6-4b54-8982-2d885e77414d None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:19:50.006506 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-cf3125e5-d7f6-4b54-8982-2d885e77414d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:50.113763 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3125e5-d7f6-4b54-8982-2d885e77414d None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:19:50.170303 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:50.189407 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527499bb-8c8a-4aec-9bf0-c19a798eed90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:19:50.190054 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527499bb-8c8a-4aec-9bf0-c19a798eed90 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:50.190358 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527499bb-8c8a-4aec-9bf0-c19a798eed90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:50.190616 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527499bb-8c8a-4aec-9bf0-c19a798eed90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:50.192854 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-527499bb-8c8a-4aec-9bf0-c19a798eed90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:50.303119 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527499bb-8c8a-4aec-9bf0-c19a798eed90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:19:50.304529 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527499bb-8c8a-4aec-9bf0-c19a798eed90 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:19:50.323331 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 742/1461] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:19:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:19:52.482365 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294d2cc-b18a-4cd2-88fb-2616b86c313c None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:52.482915 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294d2cc-b18a-4cd2-88fb-2616b86c313c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:52.483013 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294d2cc-b18a-4cd2-88fb-2616b86c313c None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:52.484373 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 720/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 06:19:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:19:52.490151 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3731b-5132-4a53-820b-b7605e274b08 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:52.490681 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3731b-5132-4a53-820b-b7605e274b08 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:52.490681 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3731b-5132-4a53-820b-b7605e274b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:52.557948 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-07a3731b-5132-4a53-820b-b7605e274b08 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:19:52.559146 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-07a3731b-5132-4a53-820b-b7605e274b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:52.598677 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:52.659277 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 743/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 06:19:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:19:52.698360 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a294d2cc-b18a-4cd2-88fb-2616b86c313c None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:19:52.699940 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a294d2cc-b18a-4cd2-88fb-2616b86c313c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:52.813253 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a294d2cc-b18a-4cd2-88fb-2616b86c313c None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:19:52.851719 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:52.888183 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13cc406-3ce4-4b63-9210-52a676bc180f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:19:52.888859 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13cc406-3ce4-4b63-9210-52a676bc180f service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:52.889125 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13cc406-3ce4-4b63-9210-52a676bc180f service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:52.889402 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13cc406-3ce4-4b63-9210-52a676bc180f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:52.891057 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e13cc406-3ce4-4b63-9210-52a676bc180f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:53.000650 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13cc406-3ce4-4b63-9210-52a676bc180f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:19:53.002088 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13cc406-3ce4-4b63-9210-52a676bc180f service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:19:53.022903 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 721/1464] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:19:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 06:19:55.093703 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf496a1-7765-4d7d-9aa1-4d38aa710f92 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:55.094246 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf496a1-7765-4d7d-9aa1-4d38aa710f92 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:55.094246 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf496a1-7765-4d7d-9aa1-4d38aa710f92 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:55.095549 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 744/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 06:19:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:19:55.101257 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02f4a5-3bc2-4015-8230-63637c6421a9 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:55.101687 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02f4a5-3bc2-4015-8230-63637c6421a9 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:55.101823 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02f4a5-3bc2-4015-8230-63637c6421a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:55.169584 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-8f02f4a5-3bc2-4015-8230-63637c6421a9 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:19:55.170821 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8f02f4a5-3bc2-4015-8230-63637c6421a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:55.189271 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:55.272320 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 722/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 06:19:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:19:55.309598 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf496a1-7765-4d7d-9aa1-4d38aa710f92 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:19:55.311184 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-9cf496a1-7765-4d7d-9aa1-4d38aa710f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:55.420024 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf496a1-7765-4d7d-9aa1-4d38aa710f92 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:19:55.491541 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:55.496242 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d2d76b-8f4b-4231-ab64-d610f0a3fdeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:19:55.496970 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d2d76b-8f4b-4231-ab64-d610f0a3fdeb service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:55.497264 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d2d76b-8f4b-4231-ab64-d610f0a3fdeb service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:55.497615 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d2d76b-8f4b-4231-ab64-d610f0a3fdeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:55.499468 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-29d2d76b-8f4b-4231-ab64-d610f0a3fdeb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:55.620158 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d2d76b-8f4b-4231-ab64-d610f0a3fdeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:19:55.622042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d2d76b-8f4b-4231-ab64-d610f0a3fdeb service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:19:55.642361 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 745/1467] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:19:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:19:57.707291 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541685cd-fa4b-4cf1-bca9-45dcb26d0be7 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:57.707818 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541685cd-fa4b-4cf1-bca9-45dcb26d0be7 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:57.707818 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541685cd-fa4b-4cf1-bca9-45dcb26d0be7 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:57.709250 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 723/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 06:19:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:19:57.714749 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561159e-eb56-48f2-a4d5-ad521d264ff6 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:57.715185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561159e-eb56-48f2-a4d5-ad521d264ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:57.715342 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561159e-eb56-48f2-a4d5-ad521d264ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:57.782026 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-b561159e-eb56-48f2-a4d5-ad521d264ff6 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:19:57.783238 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-b561159e-eb56-48f2-a4d5-ad521d264ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:57.835279 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:57.884538 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 746/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 06:19:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:19:57.922900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541685cd-fa4b-4cf1-bca9-45dcb26d0be7 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:19:57.924496 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-541685cd-fa4b-4cf1-bca9-45dcb26d0be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:58.029861 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541685cd-fa4b-4cf1-bca9-45dcb26d0be7 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:19:58.077814 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:19:58.104323 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedebdad-fc31-4244-9550-3020fe3187a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:19:58.104998 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedebdad-fc31-4244-9550-3020fe3187a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:19:58.105285 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedebdad-fc31-4244-9550-3020fe3187a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:19:58.105593 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedebdad-fc31-4244-9550-3020fe3187a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:19:58.107912 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-aedebdad-fc31-4244-9550-3020fe3187a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:19:58.217819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedebdad-fc31-4244-9550-3020fe3187a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:19:58.219166 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedebdad-fc31-4244-9550-3020fe3187a5 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:19:58.238475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 724/1470] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:19:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:20:07.882676 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=22 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:20:08.235935 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=66 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:20:17.929632 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d98997-7df8-4b78-b455-7d4069e04441 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:20:17.930199 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d98997-7df8-4b78-b455-7d4069e04441 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:20:17.930199 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d98997-7df8-4b78-b455-7d4069e04441 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:20:17.931555 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 747/1471] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:20:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:20:17.937156 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f5223-45df-4400-9efc-523eb720ea80 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:20:17.937612 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f5223-45df-4400-9efc-523eb720ea80 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:20:17.937712 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f5223-45df-4400-9efc-523eb720ea80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:20:18.006745 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-897f5223-45df-4400-9efc-523eb720ea80 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:20:18.007995 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-897f5223-45df-4400-9efc-523eb720ea80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:20:18.106715 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 725/1472] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:20:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:20:18.156107 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b551cbe-ec77-4209-813b-43afbffe0a6f None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:20:18.156553 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b551cbe-ec77-4209-813b-43afbffe0a6f None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:20:18.156697 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b551cbe-ec77-4209-813b-43afbffe0a6f None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:20:18.158158 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 748/1473] 10.222.0.40 () {54 vars in 961 bytes} [Mon Jul 18 06:20:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:20:18.163782 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc6092-e0e5-4409-86b2-b81e59e06702 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:20:18.164098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc6092-e0e5-4409-86b2-b81e59e06702 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:20:18.164386 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc6092-e0e5-4409-86b2-b81e59e06702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:20:18.241819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-8bfc6092-e0e5-4409-86b2-b81e59e06702 None None] MFA Rules not processed for user `1193330cb5644765a7abb77738929ba4`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:20:18.243015 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8bfc6092-e0e5-4409-86b2-b81e59e06702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:20:18.264126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:20:18.320248 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 726/1474] 10.222.0.40 () {58 vars in 1058 bytes} [Mon Jul 18 06:20:18 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:20:18.327637 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b551cbe-ec77-4209-813b-43afbffe0a6f None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:20:18.329211 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-2b551cbe-ec77-4209-813b-43afbffe0a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:20:18.433736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b551cbe-ec77-4209-813b-43afbffe0a6f None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:20:18.508520 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5622a41e-10eb-42a0-abaa-e7892ef775e4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1193330cb5644765a7abb77738929ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:20:18.509088 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5622a41e-10eb-42a0-abaa-e7892ef775e4 service glance] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:20:18.509622 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5622a41e-10eb-42a0-abaa-e7892ef775e4 service glance] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:20:18.509898 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5622a41e-10eb-42a0-abaa-e7892ef775e4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:20:18.511539 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-5622a41e-10eb-42a0-abaa-e7892ef775e4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:20:18.568366 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:20:18.624815 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5622a41e-10eb-42a0-abaa-e7892ef775e4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:20:18.626260 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5622a41e-10eb-42a0-abaa-e7892ef775e4 service glance] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:20:18.645632 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 749/1475] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:20:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:20:20.816308 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99432081-9674-4159-8245-a32f6bd9ca8c None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:20:20.816914 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99432081-9674-4159-8245-a32f6bd9ca8c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:20:20.817027 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99432081-9674-4159-8245-a32f6bd9ca8c None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:20:20.818331 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 727/1476] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:20:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:20:20.823785 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3a1e0-32b7-4551-bc09-29b02ace8095 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:20:20.824192 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3a1e0-32b7-4551-bc09-29b02ace8095 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:20:20.824341 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3a1e0-32b7-4551-bc09-29b02ace8095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:20:20.890067 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-b1b3a1e0-32b7-4551-bc09-29b02ace8095 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:20:20.891323 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-b1b3a1e0-32b7-4551-bc09-29b02ace8095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:20:20.991417 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 750/1477] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:20:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:20:21.014482 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99432081-9674-4159-8245-a32f6bd9ca8c None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:20:21.015994 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-99432081-9674-4159-8245-a32f6bd9ca8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:20:21.143628 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99432081-9674-4159-8245-a32f6bd9ca8c None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:20:21.192509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:20:21.237733 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcec125-18ff-434b-b371-856e306fa916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:20:21.238397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcec125-18ff-434b-b371-856e306fa916 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:20:21.238680 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcec125-18ff-434b-b371-856e306fa916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:20:21.238950 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcec125-18ff-434b-b371-856e306fa916 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:20:21.239687 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcec125-18ff-434b-b371-856e306fa916 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:20:21.241599 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: WARNING py.warnings [None req-dfcec125-18ff-434b-b371-856e306fa916 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 06:20:21.241599 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: warnings.warn(msg) Jul 18 06:20:21.241599 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: Jul 18 06:20:21.242678 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcec125-18ff-434b-b371-856e306fa916 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:20:21.262997 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 728/1478] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 18 06:20:21 2022] GET /identity/v3/projects => generated 2553 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:20:23.359715 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade4f45-9a57-4978-83fe-39c4a6bc1871 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:20:23.360290 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade4f45-9a57-4978-83fe-39c4a6bc1871 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:20:23.360290 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade4f45-9a57-4978-83fe-39c4a6bc1871 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:20:23.362527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 751/1479] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:20:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:20:23.368085 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c44e0-01a3-45d2-a044-2e499b48ed76 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:20:23.368572 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c44e0-01a3-45d2-a044-2e499b48ed76 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:20:23.368664 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c44e0-01a3-45d2-a044-2e499b48ed76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:20:23.436904 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-413c44e0-01a3-45d2-a044-2e499b48ed76 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:20:23.438150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-413c44e0-01a3-45d2-a044-2e499b48ed76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:20:23.540657 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 729/1480] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:20:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:20:23.585508 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade4f45-9a57-4978-83fe-39c4a6bc1871 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:20:23.587119 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-3ade4f45-9a57-4978-83fe-39c4a6bc1871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:20:23.598032 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:20:23.737505 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade4f45-9a57-4978-83fe-39c4a6bc1871 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:20:23.820674 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:20:23.856038 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2cfe5-f4ee-42a4-907c-71d220385654 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:20:23.856722 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2cfe5-f4ee-42a4-907c-71d220385654 service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:20:23.857012 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2cfe5-f4ee-42a4-907c-71d220385654 service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:20:23.857292 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2cfe5-f4ee-42a4-907c-71d220385654 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:20:23.858937 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ccf2cfe5-f4ee-42a4-907c-71d220385654 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:20:23.949661 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf2cfe5-f4ee-42a4-907c-71d220385654 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:20:23.951292 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf2cfe5-f4ee-42a4-907c-71d220385654 service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:20:23.971024 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 752/1481] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:20:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:20:26.218465 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49912fff-8185-4583-aa52-7bf5b8fe6908 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:20:26.219098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49912fff-8185-4583-aa52-7bf5b8fe6908 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:20:26.219098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49912fff-8185-4583-aa52-7bf5b8fe6908 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:20:26.220965 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 730/1482] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:20:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:20:26.226509 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072a44e-d980-4a0a-bfd4-0a81e34d33f2 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:20:26.227010 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072a44e-d980-4a0a-bfd4-0a81e34d33f2 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:20:26.227153 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072a44e-d980-4a0a-bfd4-0a81e34d33f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:20:26.295625 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-3072a44e-d980-4a0a-bfd4-0a81e34d33f2 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:20:26.296999 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-3072a44e-d980-4a0a-bfd4-0a81e34d33f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:20:26.344994 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:20:26.398736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 753/1483] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:20:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:20:26.435792 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49912fff-8185-4583-aa52-7bf5b8fe6908 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:20:26.437375 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-49912fff-8185-4583-aa52-7bf5b8fe6908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:20:26.462241 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:20:26.545828 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49912fff-8185-4583-aa52-7bf5b8fe6908 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:20:26.620956 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329332a6-64f2-4599-9189-239bc54febff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:20:26.621559 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329332a6-64f2-4599-9189-239bc54febff service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:20:26.621851 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329332a6-64f2-4599-9189-239bc54febff service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:20:26.622133 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329332a6-64f2-4599-9189-239bc54febff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:20:26.623861 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-329332a6-64f2-4599-9189-239bc54febff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:20:26.701951 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:20:26.737512 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329332a6-64f2-4599-9189-239bc54febff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:20:26.738966 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329332a6-64f2-4599-9189-239bc54febff service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:20:26.758132 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 731/1484] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:20:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:20:28.892524 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40962a11-5957-41c5-bec9-e6055c62efc3 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:20:28.893101 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40962a11-5957-41c5-bec9-e6055c62efc3 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:20:28.893101 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40962a11-5957-41c5-bec9-e6055c62efc3 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:20:28.894463 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 754/1485] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:20:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:20:28.900217 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b5e35b-dac6-47e8-bc75-f65ad671dfae None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:20:28.900719 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b5e35b-dac6-47e8-bc75-f65ad671dfae None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:20:28.900831 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b5e35b-dac6-47e8-bc75-f65ad671dfae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:20:28.969330 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-b7b5e35b-dac6-47e8-bc75-f65ad671dfae None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:20:28.970476 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b7b5e35b-dac6-47e8-bc75-f65ad671dfae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:20:29.071370 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 732/1486] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:20:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:20:29.109960 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40962a11-5957-41c5-bec9-e6055c62efc3 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:20:29.112311 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-40962a11-5957-41c5-bec9-e6055c62efc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:20:29.219103 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40962a11-5957-41c5-bec9-e6055c62efc3 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:20:29.296325 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:20:29.298819 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccde812-21da-4f8d-a4ed-20380f52e9fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:20:29.299532 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccde812-21da-4f8d-a4ed-20380f52e9fa service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:20:29.299822 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccde812-21da-4f8d-a4ed-20380f52e9fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:20:29.300103 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccde812-21da-4f8d-a4ed-20380f52e9fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:20:29.301838 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-8ccde812-21da-4f8d-a4ed-20380f52e9fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:20:29.413981 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccde812-21da-4f8d-a4ed-20380f52e9fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:20:29.415437 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccde812-21da-4f8d-a4ed-20380f52e9fa service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:20:29.435365 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 755/1487] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:20:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:20:31.590055 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b00293-2d50-439d-9bb8-99d74424984c None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:20:31.590602 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b00293-2d50-439d-9bb8-99d74424984c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:20:31.590602 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b00293-2d50-439d-9bb8-99d74424984c None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:20:31.591909 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 733/1488] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:20:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:20:31.597814 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9658dfb-d772-4c6e-8d04-c1741a65c522 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:20:31.598395 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9658dfb-d772-4c6e-8d04-c1741a65c522 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:20:31.598395 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9658dfb-d772-4c6e-8d04-c1741a65c522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:20:31.666007 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-c9658dfb-d772-4c6e-8d04-c1741a65c522 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:20:31.667244 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c9658dfb-d772-4c6e-8d04-c1741a65c522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:20:31.724821 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:20:31.770182 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 756/1489] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:20:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:20:31.812078 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b00293-2d50-439d-9bb8-99d74424984c None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:20:31.813694 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-48b00293-2d50-439d-9bb8-99d74424984c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:20:31.849537 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:20:31.919822 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b00293-2d50-439d-9bb8-99d74424984c None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:20:31.995465 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55063df5-52f8-4a18-a7fa-1cbaadf02fb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:20:31.996156 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55063df5-52f8-4a18-a7fa-1cbaadf02fb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:20:31.996471 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55063df5-52f8-4a18-a7fa-1cbaadf02fb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:20:31.996758 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55063df5-52f8-4a18-a7fa-1cbaadf02fb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:20:31.998514 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-55063df5-52f8-4a18-a7fa-1cbaadf02fb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:20:32.086058 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:20:32.118151 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55063df5-52f8-4a18-a7fa-1cbaadf02fb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:20:32.119889 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55063df5-52f8-4a18-a7fa-1cbaadf02fb2 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:20:32.138757 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 734/1490] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:20:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:20:34.634247 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7decb3d8-c490-4caf-a291-e22ce6ea4a6b None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:20:34.634801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7decb3d8-c490-4caf-a291-e22ce6ea4a6b None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:20:34.634801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7decb3d8-c490-4caf-a291-e22ce6ea4a6b None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:20:34.636209 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 757/1491] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 18 06:20:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:20:34.642377 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc064b12-108c-436a-bacd-7b34fab18de9 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:20:34.642834 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc064b12-108c-436a-bacd-7b34fab18de9 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:20:34.642933 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc064b12-108c-436a-bacd-7b34fab18de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:20:34.712728 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-cc064b12-108c-436a-bacd-7b34fab18de9 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:20:34.713967 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-cc064b12-108c-436a-bacd-7b34fab18de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:20:34.817480 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 735/1492] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 18 06:20:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:20:34.856617 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7decb3d8-c490-4caf-a291-e22ce6ea4a6b None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:20:34.858180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-7decb3d8-c490-4caf-a291-e22ce6ea4a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:20:34.965994 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7decb3d8-c490-4caf-a291-e22ce6ea4a6b None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:20:35.040940 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e33278-d054-4179-b35d-673d271af7f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:20:35.041613 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e33278-d054-4179-b35d-673d271af7f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:20:35.041899 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e33278-d054-4179-b35d-673d271af7f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:20:35.042189 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e33278-d054-4179-b35d-673d271af7f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:20:35.043839 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-65e33278-d054-4179-b35d-673d271af7f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:20:35.054102 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:20:35.157981 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e33278-d054-4179-b35d-673d271af7f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:20:35.159466 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e33278-d054-4179-b35d-673d271af7f9 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:20:35.178860 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 758/1493] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:20:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:20:44.815616 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=82 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:20:45.176703 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=50 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:22:12.466691 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb477b08-f069-4614-8649-489172528a0f None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:22:12.467307 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb477b08-f069-4614-8649-489172528a0f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:22:12.467307 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb477b08-f069-4614-8649-489172528a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:22:12.547693 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-eb477b08-f069-4614-8649-489172528a0f None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:22:12.548947 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-eb477b08-f069-4614-8649-489172528a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:22:12.651150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 736/1494] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:22:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:22:12.660501 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad8075-f155-40cb-9031-0293722e8947 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:22:12.661119 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad8075-f155-40cb-9031-0293722e8947 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:22:12.661119 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad8075-f155-40cb-9031-0293722e8947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:22:12.742579 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-56ad8075-f155-40cb-9031-0293722e8947 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:22:12.743762 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-56ad8075-f155-40cb-9031-0293722e8947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:22:12.845350 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 759/1495] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:22:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:22:12.850156 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb477b08-f069-4614-8649-489172528a0f None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:22:12.851409 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-eb477b08-f069-4614-8649-489172528a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:22:12.864040 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb477b08-f069-4614-8649-489172528a0f None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:22:13.047684 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:22:13.081257 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b9b140-8eb7-407a-84a6-8e62a06b9578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:22:13.081926 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9b140-8eb7-407a-84a6-8e62a06b9578 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:22:13.082212 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9b140-8eb7-407a-84a6-8e62a06b9578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:22:13.082489 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9b140-8eb7-407a-84a6-8e62a06b9578 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:22:13.083215 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b9b140-8eb7-407a-84a6-8e62a06b9578 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:22:13.083522 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b9b140-8eb7-407a-84a6-8e62a06b9578 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:22:13.084578 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: WARNING py.warnings [None req-23b9b140-8eb7-407a-84a6-8e62a06b9578 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 06:22:13.084578 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: warnings.warn(msg) Jul 18 06:22:13.084578 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: Jul 18 06:22:13.085515 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b9b140-8eb7-407a-84a6-8e62a06b9578 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:22:13.097800 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 737/1496] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:22:13.106653 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad8075-f155-40cb-9031-0293722e8947 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:22:13.116030 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad8075-f155-40cb-9031-0293722e8947 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:22:13.177461 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:22:13.211766 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f1a44d-79a9-4a4b-84cb-4f85e4bad525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:22:13.212462 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1a44d-79a9-4a4b-84cb-4f85e4bad525 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:22:13.212784 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1a44d-79a9-4a4b-84cb-4f85e4bad525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:22:13.213068 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1a44d-79a9-4a4b-84cb-4f85e4bad525 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:22:13.213761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f1a44d-79a9-4a4b-84cb-4f85e4bad525 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:22:13.216239 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: WARNING py.warnings [None req-34f1a44d-79a9-4a4b-84cb-4f85e4bad525 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 06:22:13.216239 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: warnings.warn(msg) Jul 18 06:22:13.216239 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: Jul 18 06:22:13.217609 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f1a44d-79a9-4a4b-84cb-4f85e4bad525 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:22:13.322987 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 760/1497] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:22:13 2022] POST /identity/v3/projects => generated 379 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:22:13.327670 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b9b140-8eb7-407a-84a6-8e62a06b9578 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:22:13.329244 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-23b9b140-8eb7-407a-84a6-8e62a06b9578 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:22:13.362865 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b9b140-8eb7-407a-84a6-8e62a06b9578 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:22:13.455761 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c6e301-5da0-44f2-9a6a-a3bb985b4a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:22:13.456450 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6e301-5da0-44f2-9a6a-a3bb985b4a13 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:22:13.456742 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6e301-5da0-44f2-9a6a-a3bb985b4a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:22:13.457021 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6e301-5da0-44f2-9a6a-a3bb985b4a13 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:22:13.457719 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c6e301-5da0-44f2-9a6a-a3bb985b4a13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:22:13.458727 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: WARNING py.warnings [None req-05c6e301-5da0-44f2-9a6a-a3bb985b4a13 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 06:22:13.458727 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: warnings.warn(msg) Jul 18 06:22:13.458727 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: Jul 18 06:22:13.459929 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c6e301-5da0-44f2-9a6a-a3bb985b4a13 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:22:13.601648 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 738/1498] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:22:13 2022] POST /identity/v3/users => generated 290 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:22:13.606540 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f1a44d-79a9-4a4b-84cb-4f85e4bad525 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:22:13.608233 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-34f1a44d-79a9-4a4b-84cb-4f85e4bad525 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:22:13.678652 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f1a44d-79a9-4a4b-84cb-4f85e4bad525 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:22:13.767074 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:22:13.778114 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6495cd-6618-4ccd-94e1-6ed26076d0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:22:13.778786 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6495cd-6618-4ccd-94e1-6ed26076d0d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:22:13.779071 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6495cd-6618-4ccd-94e1-6ed26076d0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:22:13.779345 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6495cd-6618-4ccd-94e1-6ed26076d0d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:22:13.780065 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6495cd-6618-4ccd-94e1-6ed26076d0d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:22:13.781138 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: WARNING py.warnings [None req-ab6495cd-6618-4ccd-94e1-6ed26076d0d9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 06:22:13.781138 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: warnings.warn(msg) Jul 18 06:22:13.781138 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: Jul 18 06:22:13.782115 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6495cd-6618-4ccd-94e1-6ed26076d0d9 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:22:13.795209 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 761/1499] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:22:13 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:22:13.799454 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c6e301-5da0-44f2-9a6a-a3bb985b4a13 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:22:13.801061 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-05c6e301-5da0-44f2-9a6a-a3bb985b4a13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:22:13.812537 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c6e301-5da0-44f2-9a6a-a3bb985b4a13 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:22:13.837747 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:22:13.908322 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77964a2-0f50-4652-8e31-ea3fe3ffecda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:22:13.909024 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77964a2-0f50-4652-8e31-ea3fe3ffecda admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:22:13.909319 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77964a2-0f50-4652-8e31-ea3fe3ffecda admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:22:13.909578 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77964a2-0f50-4652-8e31-ea3fe3ffecda admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:22:13.910300 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77964a2-0f50-4652-8e31-ea3fe3ffecda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:22:13.911342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: WARNING py.warnings [None req-d77964a2-0f50-4652-8e31-ea3fe3ffecda admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 06:22:13.911342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: warnings.warn(msg) Jul 18 06:22:13.911342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: Jul 18 06:22:13.912172 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77964a2-0f50-4652-8e31-ea3fe3ffecda admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:22:13.924903 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 739/1500] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:22:13 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:22:13.929102 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6495cd-6618-4ccd-94e1-6ed26076d0d9 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:22:13.930654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ab6495cd-6618-4ccd-94e1-6ed26076d0d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:22:13.965827 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6495cd-6618-4ccd-94e1-6ed26076d0d9 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:22:14.178219 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f50cab-31b9-4e9a-bb39-d33499835584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:22:14.179027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f50cab-31b9-4e9a-bb39-d33499835584 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:22:14.179310 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f50cab-31b9-4e9a-bb39-d33499835584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:22:14.179595 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f50cab-31b9-4e9a-bb39-d33499835584 admin admin] PATH_INFO: `/v3/projects/60715df4b7f04eafaf6934c23d7df832/users/944b90cc499f4d109b832c6bbf97bb39/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:22:14.202729 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f50cab-31b9-4e9a-bb39-d33499835584 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60715df4b7f04eafaf6934c23d7df832, user_id=944b90cc499f4d109b832c6bbf97bb39, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:22:14.203776 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: WARNING py.warnings [None req-49f50cab-31b9-4e9a-bb39-d33499835584 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 06:22:14.203776 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: warnings.warn(msg) Jul 18 06:22:14.203776 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: Jul 18 06:22:14.205758 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f50cab-31b9-4e9a-bb39-d33499835584 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:22:14.535973 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 762/1501] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:22:13 2022] PUT /identity/v3/projects/60715df4b7f04eafaf6934c23d7df832/users/944b90cc499f4d109b832c6bbf97bb39/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:22:14.548831 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda25b0-c065-4500-8817-ff5deda99e8d None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:22:14.549166 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda25b0-c065-4500-8817-ff5deda99e8d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:22:14.549483 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda25b0-c065-4500-8817-ff5deda99e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:22:14.604707 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-5bda25b0-c065-4500-8817-ff5deda99e8d None None] MFA Rules not processed for user `944b90cc499f4d109b832c6bbf97bb39`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:22:14.606530 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-5bda25b0-c065-4500-8817-ff5deda99e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:22:14.686031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 740/1502] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:22:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:22:14.699266 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91329a5-d67b-452f-9b9b-38144717a078 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:22:14.699701 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91329a5-d67b-452f-9b9b-38144717a078 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:22:14.699832 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91329a5-d67b-452f-9b9b-38144717a078 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:22:14.701227 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 763/1503] 10.222.0.40 () {54 vars in 964 bytes} [Mon Jul 18 06:22:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:22:14.706203 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5206b-6264-4edb-af8e-82c61e4acec9 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:22:14.706499 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5206b-6264-4edb-af8e-82c61e4acec9 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:22:14.706783 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5206b-6264-4edb-af8e-82c61e4acec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:22:14.736928 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:22:14.780353 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-23b5206b-6264-4edb-af8e-82c61e4acec9 None None] MFA Rules not processed for user `5af3a4a1bc1f4c4e9653d0a790f12606`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:22:14.781587 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-23b5206b-6264-4edb-af8e-82c61e4acec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:22:14.857220 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 741/1504] 10.222.0.40 () {58 vars in 1061 bytes} [Mon Jul 18 06:22:14 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:22:14.865012 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91329a5-d67b-452f-9b9b-38144717a078 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:22:14.866708 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-b91329a5-d67b-452f-9b9b-38144717a078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:22:14.917381 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:22:14.970135 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91329a5-d67b-452f-9b9b-38144717a078 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:22:15.046095 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9800a6a-0c91-4b95-b165-3fc4276be38b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:22:15.046759 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9800a6a-0c91-4b95-b165-3fc4276be38b service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:22:15.047044 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9800a6a-0c91-4b95-b165-3fc4276be38b service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:22:15.047326 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9800a6a-0c91-4b95-b165-3fc4276be38b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:22:15.049005 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e9800a6a-0c91-4b95-b165-3fc4276be38b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:22:15.149781 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9800a6a-0c91-4b95-b165-3fc4276be38b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:22:15.151234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9800a6a-0c91-4b95-b165-3fc4276be38b service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:22:15.155489 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:22:15.171710 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 764/1505] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:22:14 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 06:22:15.213249 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cade2b2-1604-4f06-99ba-1dc74b9d0309 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:22:15.217088 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cade2b2-1604-4f06-99ba-1dc74b9d0309 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:22:15.217088 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cade2b2-1604-4f06-99ba-1dc74b9d0309 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:22:15.217088 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 742/1506] 10.222.0.40 () {52 vars in 829 bytes} [Mon Jul 18 06:22:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:22:15.220608 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9800a6a-0c91-4b95-b165-3fc4276be38b service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:22:15.221861 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e9800a6a-0c91-4b95-b165-3fc4276be38b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:22:15.324287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9800a6a-0c91-4b95-b165-3fc4276be38b service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:22:15.416787 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77bfdd5-8bce-4794-99af-5d3e3ff7f291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:22:15.418096 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77bfdd5-8bce-4794-99af-5d3e3ff7f291 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:22:15.418347 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77bfdd5-8bce-4794-99af-5d3e3ff7f291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:22:15.418618 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77bfdd5-8bce-4794-99af-5d3e3ff7f291 admin admin] PATH_INFO: `/v3/users/944b90cc499f4d109b832c6bbf97bb39` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:22:15.430763 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77bfdd5-8bce-4794-99af-5d3e3ff7f291 admin admin] RBAC: Authorizing `identity:delete_user(user_id=944b90cc499f4d109b832c6bbf97bb39)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:22:15.431786 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: WARNING py.warnings [None req-f77bfdd5-8bce-4794-99af-5d3e3ff7f291 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 06:22:15.431786 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: warnings.warn(msg) Jul 18 06:22:15.431786 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: Jul 18 06:22:15.433019 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77bfdd5-8bce-4794-99af-5d3e3ff7f291 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:22:15.443313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:22:15.581172 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-f77bfdd5-8bce-4794-99af-5d3e3ff7f291 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '944b90cc499f4d109b832c6bbf97bb39'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:22:15.593946 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-f77bfdd5-8bce-4794-99af-5d3e3ff7f291 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '944b90cc499f4d109b832c6bbf97bb39'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:22:15.594528 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-f77bfdd5-8bce-4794-99af-5d3e3ff7f291 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '944b90cc499f4d109b832c6bbf97bb39'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:22:15.594955 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-f77bfdd5-8bce-4794-99af-5d3e3ff7f291 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '944b90cc499f4d109b832c6bbf97bb39'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:22:15.614008 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-f77bfdd5-8bce-4794-99af-5d3e3ff7f291 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '944b90cc499f4d109b832c6bbf97bb39'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:22:15.675679 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 765/1507] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:22:15 2022] DELETE /identity/v3/users/944b90cc499f4d109b832c6bbf97bb39 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:22:15.680089 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b055b86-42b7-48b2-9ccc-6104ee0839b2 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:22:15.680599 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b055b86-42b7-48b2-9ccc-6104ee0839b2 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:22:15.680700 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b055b86-42b7-48b2-9ccc-6104ee0839b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:22:15.737034 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-1b055b86-42b7-48b2-9ccc-6104ee0839b2 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:22:15.738389 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-1b055b86-42b7-48b2-9ccc-6104ee0839b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:22:15.814205 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:22:15.826941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 743/1508] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:22:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:22:15.836735 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77bfdd5-8bce-4794-99af-5d3e3ff7f291 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:22:15.837998 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-f77bfdd5-8bce-4794-99af-5d3e3ff7f291 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:22:15.938521 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77bfdd5-8bce-4794-99af-5d3e3ff7f291 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:22:15.994152 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:22:16.014976 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e7856a-3b7e-4c20-9b2c-a5a802984e41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:22:16.015636 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7856a-3b7e-4c20-9b2c-a5a802984e41 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:22:16.015925 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7856a-3b7e-4c20-9b2c-a5a802984e41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:22:16.016208 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7856a-3b7e-4c20-9b2c-a5a802984e41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:22:16.017967 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d4e7856a-3b7e-4c20-9b2c-a5a802984e41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:22:16.127609 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e7856a-3b7e-4c20-9b2c-a5a802984e41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:22:16.129122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e7856a-3b7e-4c20-9b2c-a5a802984e41 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:22:16.147698 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 766/1509] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:22:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:22:16.614228 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b055b86-42b7-48b2-9ccc-6104ee0839b2 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:22:16.615488 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-1b055b86-42b7-48b2-9ccc-6104ee0839b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:22:16.627605 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b055b86-42b7-48b2-9ccc-6104ee0839b2 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:22:16.729548 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603fa990-985e-448d-94b2-e5f832d405da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:22:16.731134 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603fa990-985e-448d-94b2-e5f832d405da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:22:16.731365 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603fa990-985e-448d-94b2-e5f832d405da admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:22:16.731644 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603fa990-985e-448d-94b2-e5f832d405da admin admin] PATH_INFO: `/v3/projects/60715df4b7f04eafaf6934c23d7df832` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:22:16.742023 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603fa990-985e-448d-94b2-e5f832d405da admin admin] RBAC: Authorizing `identity:delete_project(project_id=60715df4b7f04eafaf6934c23d7df832)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:22:16.743048 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: WARNING py.warnings [None req-603fa990-985e-448d-94b2-e5f832d405da admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 06:22:16.743048 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: warnings.warn(msg) Jul 18 06:22:16.743048 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: Jul 18 06:22:16.744265 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603fa990-985e-448d-94b2-e5f832d405da admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:22:16.858013 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-603fa990-985e-448d-94b2-e5f832d405da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60715df4b7f04eafaf6934c23d7df832', 'request_id': 'req-603fa990-985e-448d-94b2-e5f832d405da'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:22:16.864175 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-603fa990-985e-448d-94b2-e5f832d405da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60715df4b7f04eafaf6934c23d7df832', 'request_id': 'req-603fa990-985e-448d-94b2-e5f832d405da'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:22:16.931464 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-603fa990-985e-448d-94b2-e5f832d405da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60715df4b7f04eafaf6934c23d7df832', 'request_id': 'req-603fa990-985e-448d-94b2-e5f832d405da'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:22:16.962108 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-603fa990-985e-448d-94b2-e5f832d405da admin admin] The token cache is being invalidate because project 60715df4b7f04eafaf6934c23d7df832 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:22:16.962401 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-603fa990-985e-448d-94b2-e5f832d405da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:22:16.962833 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-603fa990-985e-448d-94b2-e5f832d405da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:22:16.964276 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 744/1510] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:22:16 2022] DELETE /identity/v3/projects/60715df4b7f04eafaf6934c23d7df832 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:22:26.145447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=62 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:22:26.960048 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:23:56.916454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78345a89-c27f-463b-a6c3-00268e86c7bc None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:23:56.917128 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78345a89-c27f-463b-a6c3-00268e86c7bc None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:23:56.917128 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78345a89-c27f-463b-a6c3-00268e86c7bc None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:23:56.919064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 767/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 06:23:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:23:56.924350 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee9a71-b8db-4f08-9a39-71d513ad6b1a None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:23:56.924877 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee9a71-b8db-4f08-9a39-71d513ad6b1a None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:23:56.924987 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee9a71-b8db-4f08-9a39-71d513ad6b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:23:57.007710 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-aaee9a71-b8db-4f08-9a39-71d513ad6b1a None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:23:57.008876 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-aaee9a71-b8db-4f08-9a39-71d513ad6b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:23:57.115731 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 745/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 06:23:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:23:57.269813 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78345a89-c27f-463b-a6c3-00268e86c7bc None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:23:57.272005 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-78345a89-c27f-463b-a6c3-00268e86c7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:23:57.399353 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78345a89-c27f-463b-a6c3-00268e86c7bc None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:23:57.405064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29156932-319a-4f64-8ca7-045b8cb6b1f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:23:57.405956 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29156932-319a-4f64-8ca7-045b8cb6b1f8 service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:23:57.406246 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29156932-319a-4f64-8ca7-045b8cb6b1f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:23:57.406527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29156932-319a-4f64-8ca7-045b8cb6b1f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:23:57.415487 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29156932-319a-4f64-8ca7-045b8cb6b1f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:23:57.417662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29156932-319a-4f64-8ca7-045b8cb6b1f8 service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:23:57.428288 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 768/1513] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:23:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:23:58.287088 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09d416-2ee5-414a-b730-fe7e35586b2b None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:23:58.287698 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09d416-2ee5-414a-b730-fe7e35586b2b None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:23:58.287698 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09d416-2ee5-414a-b730-fe7e35586b2b None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:23:58.289085 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 746/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 06:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:23:58.294850 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572862f4-eb99-4969-af98-a938b1f0bcf7 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:23:58.295263 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572862f4-eb99-4969-af98-a938b1f0bcf7 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:23:58.295453 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572862f4-eb99-4969-af98-a938b1f0bcf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:23:58.314358 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-572862f4-eb99-4969-af98-a938b1f0bcf7 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:23:58.315665 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-572862f4-eb99-4969-af98-a938b1f0bcf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:23:58.323722 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 769/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 06:23:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:23:58.475585 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b09d416-2ee5-414a-b730-fe7e35586b2b None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:23:58.491132 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b09d416-2ee5-414a-b730-fe7e35586b2b None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:23:58.496776 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d09152-d385-4d39-9dcc-075120fee19d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:23:58.497444 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d09152-d385-4d39-9dcc-075120fee19d service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:23:58.497736 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d09152-d385-4d39-9dcc-075120fee19d service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:23:58.498019 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d09152-d385-4d39-9dcc-075120fee19d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:23:58.507112 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d09152-d385-4d39-9dcc-075120fee19d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:23:58.508856 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d09152-d385-4d39-9dcc-075120fee19d service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:23:58.518751 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 747/1516] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:23:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:23:59.073041 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d2723-0f58-448e-8ce5-962902a46140 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:23:59.073359 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d2723-0f58-448e-8ce5-962902a46140 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:23:59.073594 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d2723-0f58-448e-8ce5-962902a46140 None None] PATH_INFO: `/v3` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:23:59.074924 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 770/1517] 10.222.0.17 () {54 vars in 925 bytes} [Mon Jul 18 06:23:59 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:23:59.081052 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c4bcba-e4df-4e9b-b997-851f00fb71cc None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:23:59.081348 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c4bcba-e4df-4e9b-b997-851f00fb71cc None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:23:59.081634 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c4bcba-e4df-4e9b-b997-851f00fb71cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:23:59.140746 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-d3c4bcba-e4df-4e9b-b997-851f00fb71cc None None] MFA Rules not processed for user `a3560534427544cdabfd0ba702a0a165`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:23:59.141981 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-d3c4bcba-e4df-4e9b-b997-851f00fb71cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:23:59.159830 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:23:59.219804 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 748/1518] 10.222.0.17 () {58 vars in 1015 bytes} [Mon Jul 18 06:23:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:24:08.309486 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=56 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:24:08.881342 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3303c-a6ec-4f2a-ba6c-83844d779e00 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:24:08.882282 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3303c-a6ec-4f2a-ba6c-83844d779e00 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:24:08.882282 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3303c-a6ec-4f2a-ba6c-83844d779e00 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:24:08.883258 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 771/1519] 10.222.0.40 () {54 vars in 961 bytes} [Mon Jul 18 06:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:24:08.889149 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f72acf-ab61-4bc7-960c-89006ea2dab6 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:24:08.889667 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f72acf-ab61-4bc7-960c-89006ea2dab6 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:24:08.889852 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f72acf-ab61-4bc7-960c-89006ea2dab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:24:08.910232 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-b0f72acf-ab61-4bc7-960c-89006ea2dab6 None None] MFA Rules not processed for user `a3560534427544cdabfd0ba702a0a165`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:24:08.911512 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b0f72acf-ab61-4bc7-960c-89006ea2dab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:24:08.920540 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 749/1520] 10.222.0.40 () {58 vars in 1058 bytes} [Mon Jul 18 06:24:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:24:08.928168 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3303c-a6ec-4f2a-ba6c-83844d779e00 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:24:08.942658 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3303c-a6ec-4f2a-ba6c-83844d779e00 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:24:08.947760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdcb6ae-4419-43c8-9575-32d1a29129cd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3560534427544cdabfd0ba702a0a165', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:24:08.948454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdcb6ae-4419-43c8-9575-32d1a29129cd service placement] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:24:08.948809 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdcb6ae-4419-43c8-9575-32d1a29129cd service placement] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:24:08.949165 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdcb6ae-4419-43c8-9575-32d1a29129cd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:24:08.959263 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdcb6ae-4419-43c8-9575-32d1a29129cd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:24:08.960900 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdcb6ae-4419-43c8-9575-32d1a29129cd service placement] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:24:08.971521 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 772/1521] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:24:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:24:09.765698 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f1b97-a97f-47eb-9bb8-cfd0133a9b08 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:24:09.766030 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f1b97-a97f-47eb-9bb8-cfd0133a9b08 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:24:09.766262 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f1b97-a97f-47eb-9bb8-cfd0133a9b08 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:24:09.767802 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 750/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 06:24:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:24:09.772829 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f14212-4953-48e3-86f7-15bbc41d9c6f None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:24:09.773258 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f14212-4953-48e3-86f7-15bbc41d9c6f None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:24:09.773359 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f14212-4953-48e3-86f7-15bbc41d9c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:24:09.791884 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-c6f14212-4953-48e3-86f7-15bbc41d9c6f None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:24:09.793324 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c6f14212-4953-48e3-86f7-15bbc41d9c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:24:09.801217 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 773/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 06:24:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:24:09.957440 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f1b97-a97f-47eb-9bb8-cfd0133a9b08 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:24:09.972309 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f1b97-a97f-47eb-9bb8-cfd0133a9b08 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:24:09.977788 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08c760-f13b-4364-84bb-3838aa55e42c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:24:09.978432 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08c760-f13b-4364-84bb-3838aa55e42c service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:24:09.978722 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08c760-f13b-4364-84bb-3838aa55e42c service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:24:09.979003 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08c760-f13b-4364-84bb-3838aa55e42c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:24:09.988259 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b08c760-f13b-4364-84bb-3838aa55e42c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:24:09.989872 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b08c760-f13b-4364-84bb-3838aa55e42c service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:24:10.000241 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 751/1524] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:24:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:24:10.819952 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac010c0f-860e-45ec-82d6-a75c95e0bc21 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:24:10.820536 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac010c0f-860e-45ec-82d6-a75c95e0bc21 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:24:10.820536 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac010c0f-860e-45ec-82d6-a75c95e0bc21 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:24:10.821888 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 774/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 06:24:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:24:10.827332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142a0e9-d430-411d-826f-209a42fe28b6 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:24:10.827628 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142a0e9-d430-411d-826f-209a42fe28b6 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:24:10.827908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142a0e9-d430-411d-826f-209a42fe28b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:24:10.846449 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-2142a0e9-d430-411d-826f-209a42fe28b6 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:24:10.847657 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-2142a0e9-d430-411d-826f-209a42fe28b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:24:10.855735 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 752/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 06:24:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:24:11.013000 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac010c0f-860e-45ec-82d6-a75c95e0bc21 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:24:11.027905 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac010c0f-860e-45ec-82d6-a75c95e0bc21 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:24:11.033523 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4cd8e0-bf46-439d-940f-3697b94e2a7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:24:11.034165 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4cd8e0-bf46-439d-940f-3697b94e2a7f service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:24:11.034454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4cd8e0-bf46-439d-940f-3697b94e2a7f service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:24:11.034736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4cd8e0-bf46-439d-940f-3697b94e2a7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:24:11.043340 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4cd8e0-bf46-439d-940f-3697b94e2a7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:24:11.044984 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4cd8e0-bf46-439d-940f-3697b94e2a7f service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:24:11.054686 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 775/1527] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:24:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:24:15.078008 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39acfcfa-bdaa-48b7-b60c-28d3ab2aca03 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:24:15.078534 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39acfcfa-bdaa-48b7-b60c-28d3ab2aca03 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:24:15.078534 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39acfcfa-bdaa-48b7-b60c-28d3ab2aca03 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:24:15.079965 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 753/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 06:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:24:15.086027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd961343-8fcb-4a27-b045-4fd47c9d48b0 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:24:15.086512 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd961343-8fcb-4a27-b045-4fd47c9d48b0 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:24:15.086646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd961343-8fcb-4a27-b045-4fd47c9d48b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:24:15.105842 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-cd961343-8fcb-4a27-b045-4fd47c9d48b0 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:24:15.107768 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-cd961343-8fcb-4a27-b045-4fd47c9d48b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:24:15.115953 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 776/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 06:24:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:24:15.156885 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39acfcfa-bdaa-48b7-b60c-28d3ab2aca03 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:24:15.158948 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-39acfcfa-bdaa-48b7-b60c-28d3ab2aca03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:24:15.245553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39acfcfa-bdaa-48b7-b60c-28d3ab2aca03 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:24:15.249266 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c543a4cb-1a1f-42ce-8ad8-a5e0616d9281 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:24:15.249911 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543a4cb-1a1f-42ce-8ad8-a5e0616d9281 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:24:15.250230 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543a4cb-1a1f-42ce-8ad8-a5e0616d9281 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:24:15.250511 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543a4cb-1a1f-42ce-8ad8-a5e0616d9281 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:24:15.260078 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c543a4cb-1a1f-42ce-8ad8-a5e0616d9281 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:24:15.262240 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c543a4cb-1a1f-42ce-8ad8-a5e0616d9281 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:24:15.271935 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 754/1530] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:24:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:24:17.297743 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59721f1f-7839-467b-ba90-733b0834e285 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:24:17.298314 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59721f1f-7839-467b-ba90-733b0834e285 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:24:17.298314 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59721f1f-7839-467b-ba90-733b0834e285 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:24:17.299690 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 777/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 06:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:24:17.304889 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c6459-4767-4ce3-9e71-4934813b4212 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:24:17.305340 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c6459-4767-4ce3-9e71-4934813b4212 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:24:17.305458 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c6459-4767-4ce3-9e71-4934813b4212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:24:17.324989 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-9f0c6459-4767-4ce3-9e71-4934813b4212 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:24:17.326196 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-9f0c6459-4767-4ce3-9e71-4934813b4212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:24:17.334197 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 755/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 06:24:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:24:17.371094 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59721f1f-7839-467b-ba90-733b0834e285 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:24:17.382375 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59721f1f-7839-467b-ba90-733b0834e285 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:24:17.385887 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d55dd-e6c1-4ace-9f24-38e5493285f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:24:17.387330 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d55dd-e6c1-4ace-9f24-38e5493285f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:24:17.387797 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d55dd-e6c1-4ace-9f24-38e5493285f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:24:17.388082 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d55dd-e6c1-4ace-9f24-38e5493285f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:24:17.397276 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d55dd-e6c1-4ace-9f24-38e5493285f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:24:17.398731 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d55dd-e6c1-4ace-9f24-38e5493285f5 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:24:17.408850 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 778/1533] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:24:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:24:20.952339 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327dffd8-2fec-4781-9b8c-492a0d4fe265 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:24:20.952896 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327dffd8-2fec-4781-9b8c-492a0d4fe265 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:24:20.952896 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327dffd8-2fec-4781-9b8c-492a0d4fe265 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:24:20.954197 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 756/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 06:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:24:20.959347 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d75f1e-d13d-4ffd-83db-fced4e7b2057 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:24:20.959735 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d75f1e-d13d-4ffd-83db-fced4e7b2057 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:24:20.959914 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d75f1e-d13d-4ffd-83db-fced4e7b2057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:24:20.978254 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-08d75f1e-d13d-4ffd-83db-fced4e7b2057 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:24:20.979457 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-08d75f1e-d13d-4ffd-83db-fced4e7b2057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:24:20.987986 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 779/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 06:24:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:24:21.296351 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327dffd8-2fec-4781-9b8c-492a0d4fe265 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:24:21.298050 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-327dffd8-2fec-4781-9b8c-492a0d4fe265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:24:21.363488 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:24:21.377790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327dffd8-2fec-4781-9b8c-492a0d4fe265 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:24:21.386618 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25d573-4ca9-436e-ad43-4c5be05bffdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:24:21.386618 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25d573-4ca9-436e-ad43-4c5be05bffdb service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:24:21.386618 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25d573-4ca9-436e-ad43-4c5be05bffdb service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:24:21.386618 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25d573-4ca9-436e-ad43-4c5be05bffdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:24:21.392633 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec25d573-4ca9-436e-ad43-4c5be05bffdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:24:21.394077 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec25d573-4ca9-436e-ad43-4c5be05bffdb service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:24:21.404092 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 757/1536] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:24:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:24:23.536840 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302d732-9e21-41dd-bc9a-1a17947b8f63 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:24:23.537405 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302d732-9e21-41dd-bc9a-1a17947b8f63 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:24:23.537405 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302d732-9e21-41dd-bc9a-1a17947b8f63 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:24:23.538832 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 780/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 06:24:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:24:23.544543 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f72187-579b-494d-b275-cf27b7b70e70 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:24:23.545081 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f72187-579b-494d-b275-cf27b7b70e70 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:24:23.545179 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f72187-579b-494d-b275-cf27b7b70e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:24:23.564552 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-85f72187-579b-494d-b275-cf27b7b70e70 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:24:23.565779 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-85f72187-579b-494d-b275-cf27b7b70e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:24:23.575001 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 758/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 06:24:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:24:23.892899 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302d732-9e21-41dd-bc9a-1a17947b8f63 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:24:23.894722 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-a302d732-9e21-41dd-bc9a-1a17947b8f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:24:23.908527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302d732-9e21-41dd-bc9a-1a17947b8f63 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:24:23.912059 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf30a02-6785-456b-ba99-97b208789106 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:24:23.912748 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf30a02-6785-456b-ba99-97b208789106 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:24:23.913039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf30a02-6785-456b-ba99-97b208789106 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:24:23.913319 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf30a02-6785-456b-ba99-97b208789106 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:24:23.924276 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf30a02-6785-456b-ba99-97b208789106 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:24:23.925750 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf30a02-6785-456b-ba99-97b208789106 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:24:23.935580 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 781/1539] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:24:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:24:33.559522 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=11 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:24:33.931113 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=33 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:25:59.189596 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99aca0b-be5c-4dec-95df-b698a178733d None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:25:59.190525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99aca0b-be5c-4dec-95df-b698a178733d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:25:59.190525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99aca0b-be5c-4dec-95df-b698a178733d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:25:59.217204 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-c99aca0b-be5c-4dec-95df-b698a178733d None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:25:59.218486 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-c99aca0b-be5c-4dec-95df-b698a178733d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:25:59.231974 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 759/1540] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:25:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:25:59.236929 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacfd6c3-8a44-45a4-a449-11c7c2c5117a None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:25:59.236929 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacfd6c3-8a44-45a4-a449-11c7c2c5117a None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:25:59.236929 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacfd6c3-8a44-45a4-a449-11c7c2c5117a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:25:59.242592 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d87fa9-6347-4e56-a666-8c447b8944e4 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:25:59.242946 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d87fa9-6347-4e56-a666-8c447b8944e4 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:25:59.243164 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d87fa9-6347-4e56-a666-8c447b8944e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:25:59.269993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-dacfd6c3-8a44-45a4-a449-11c7c2c5117a None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:25:59.272889 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-dacfd6c3-8a44-45a4-a449-11c7c2c5117a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:25:59.288318 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 782/1541] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:25:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:25:59.290410 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-61d87fa9-6347-4e56-a666-8c447b8944e4 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:25:59.291699 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-61d87fa9-6347-4e56-a666-8c447b8944e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:25:59.299590 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb2aea-0813-4d03-b9ac-48c39ab690ec None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:25:59.300334 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb2aea-0813-4d03-b9ac-48c39ab690ec None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:25:59.301038 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb2aea-0813-4d03-b9ac-48c39ab690ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:25:59.314454 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 760/1542] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:25:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:25:59.320432 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d87fa9-6347-4e56-a666-8c447b8944e4 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:25:59.326488 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-95fb2aea-0813-4d03-b9ac-48c39ab690ec None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:25:59.328146 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-95fb2aea-0813-4d03-b9ac-48c39ab690ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:25:59.351862 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d87fa9-6347-4e56-a666-8c447b8944e4 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:25:59.353700 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 783/1543] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:25:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:25:59.359774 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb2aea-0813-4d03-b9ac-48c39ab690ec None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:25:59.361507 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398f932a-b906-454b-8287-f69444b42be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:25:59.362270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f932a-b906-454b-8287-f69444b42be4 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:25:59.362574 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f932a-b906-454b-8287-f69444b42be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:25:59.362845 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f932a-b906-454b-8287-f69444b42be4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:25:59.363636 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398f932a-b906-454b-8287-f69444b42be4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:25:59.363953 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398f932a-b906-454b-8287-f69444b42be4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:25:59.366398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398f932a-b906-454b-8287-f69444b42be4 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:25:59.378982 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb2aea-0813-4d03-b9ac-48c39ab690ec None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:25:59.390555 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198592fe-01fa-471c-b074-b2eaec684d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:25:59.391818 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198592fe-01fa-471c-b074-b2eaec684d13 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:25:59.392342 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198592fe-01fa-471c-b074-b2eaec684d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:25:59.392971 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198592fe-01fa-471c-b074-b2eaec684d13 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:25:59.394026 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198592fe-01fa-471c-b074-b2eaec684d13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:25:59.394820 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198592fe-01fa-471c-b074-b2eaec684d13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:25:59.397107 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: WARNING py.warnings [None req-198592fe-01fa-471c-b074-b2eaec684d13 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 06:25:59.397107 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: warnings.warn(msg) Jul 18 06:25:59.397107 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: Jul 18 06:25:59.398586 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198592fe-01fa-471c-b074-b2eaec684d13 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:25:59.402967 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 761/1544] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:25:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:25:59.404796 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24dffa7-862c-4c8c-a779-449b360cde43 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:25:59.405136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24dffa7-862c-4c8c-a779-449b360cde43 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:25:59.405427 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24dffa7-862c-4c8c-a779-449b360cde43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:25:59.420108 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 784/1545] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:25:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:25:59.421551 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198592fe-01fa-471c-b074-b2eaec684d13 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:25:59.441560 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198592fe-01fa-471c-b074-b2eaec684d13 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:25:59.450281 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-e24dffa7-862c-4c8c-a779-449b360cde43 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:25:59.452507 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e24dffa7-862c-4c8c-a779-449b360cde43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:25:59.461407 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea1aff5-e47e-43c7-8b8c-61824cd90726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:25:59.462141 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1aff5-e47e-43c7-8b8c-61824cd90726 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:25:59.462420 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1aff5-e47e-43c7-8b8c-61824cd90726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:25:59.462706 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1aff5-e47e-43c7-8b8c-61824cd90726 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:25:59.463485 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea1aff5-e47e-43c7-8b8c-61824cd90726 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:25:59.465795 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea1aff5-e47e-43c7-8b8c-61824cd90726 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:25:59.500458 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 762/1546] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:25:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:25:59.500458 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24dffa7-862c-4c8c-a779-449b360cde43 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:25:59.533926 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24dffa7-862c-4c8c-a779-449b360cde43 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:25:59.559563 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112177a9-b455-46fd-aafc-8648ce5c34f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:25:59.561551 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112177a9-b455-46fd-aafc-8648ce5c34f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:25:59.562243 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112177a9-b455-46fd-aafc-8648ce5c34f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:25:59.562773 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112177a9-b455-46fd-aafc-8648ce5c34f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:25:59.563775 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112177a9-b455-46fd-aafc-8648ce5c34f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:25:59.565905 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: WARNING py.warnings [None req-112177a9-b455-46fd-aafc-8648ce5c34f5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 06:25:59.565905 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: warnings.warn(msg) Jul 18 06:25:59.565905 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: Jul 18 06:25:59.569607 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112177a9-b455-46fd-aafc-8648ce5c34f5 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:25:59.662719 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 785/1547] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:25:59 2022] POST /identity/v3/projects => generated 377 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:25:59.664876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be958ebd-b3bf-44be-a0b5-bdaed38d2f90 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:25:59.665894 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be958ebd-b3bf-44be-a0b5-bdaed38d2f90 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:25:59.666453 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be958ebd-b3bf-44be-a0b5-bdaed38d2f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:25:59.690220 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 763/1548] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:25:59 2022] POST /identity/v3/projects => generated 373 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:25:59.696718 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfdca4e-e25e-4f79-9de9-3098ccfd9da4 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:25:59.696718 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfdca4e-e25e-4f79-9de9-3098ccfd9da4 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:25:59.696718 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfdca4e-e25e-4f79-9de9-3098ccfd9da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:25:59.712095 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-be958ebd-b3bf-44be-a0b5-bdaed38d2f90 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:25:59.717394 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-be958ebd-b3bf-44be-a0b5-bdaed38d2f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:25:59.743990 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-5cfdca4e-e25e-4f79-9de9-3098ccfd9da4 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:25:59.745279 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-5cfdca4e-e25e-4f79-9de9-3098ccfd9da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:25:59.847299 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 786/1549] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:25:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:25:59.850217 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82acbc-c8d6-49b7-88e7-a94f32239ea9 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:25:59.850538 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82acbc-c8d6-49b7-88e7-a94f32239ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:25:59.850891 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82acbc-c8d6-49b7-88e7-a94f32239ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:25:59.878425 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-1f82acbc-c8d6-49b7-88e7-a94f32239ea9 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:25:59.879995 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-1f82acbc-c8d6-49b7-88e7-a94f32239ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:25:59.880151 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 764/1550] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:25:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:25:59.887589 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfdca4e-e25e-4f79-9de9-3098ccfd9da4 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:25:59.900846 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 787/1551] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:25:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:25:59.901717 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82acbc-c8d6-49b7-88e7-a94f32239ea9 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:25:59.903641 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfdca4e-e25e-4f79-9de9-3098ccfd9da4 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:25:59.912920 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38c2bf1-943d-4041-b253-a78f71810730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:25:59.913151 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82acbc-c8d6-49b7-88e7-a94f32239ea9 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:25:59.914105 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c2bf1-943d-4041-b253-a78f71810730 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:25:59.914674 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c2bf1-943d-4041-b253-a78f71810730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:25:59.915189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c2bf1-943d-4041-b253-a78f71810730 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:25:59.916545 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38c2bf1-943d-4041-b253-a78f71810730 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:25:59.917951 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d802b80d-fa11-4677-9d2b-b8a16acaab0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:25:59.918653 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d802b80d-fa11-4677-9d2b-b8a16acaab0a admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:25:59.919223 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d802b80d-fa11-4677-9d2b-b8a16acaab0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:25:59.919457 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d802b80d-fa11-4677-9d2b-b8a16acaab0a admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:25:59.920161 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38c2bf1-943d-4041-b253-a78f71810730 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:25:59.920291 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d802b80d-fa11-4677-9d2b-b8a16acaab0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:25:59.921248 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: WARNING py.warnings [None req-d802b80d-fa11-4677-9d2b-b8a16acaab0a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 06:25:59.921248 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: warnings.warn(msg) Jul 18 06:25:59.921248 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: Jul 18 06:25:59.922460 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d802b80d-fa11-4677-9d2b-b8a16acaab0a admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:00.139320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 788/1552] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:25:59 2022] POST /identity/v3/users => generated 287 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:00.139630 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 765/1553] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:25:59 2022] POST /identity/v3/users => generated 289 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:00.140483 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38c2bf1-943d-4041-b253-a78f71810730 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:00.141546 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5590249-f67f-43c7-b267-2eac2d6d7466 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:00.142051 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5590249-f67f-43c7-b267-2eac2d6d7466 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:00.142723 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5590249-f67f-43c7-b267-2eac2d6d7466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:00.153041 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38c2bf1-943d-4041-b253-a78f71810730 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:00.158157 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9114e49b-59dc-46e9-83bf-ecbc0564bea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:00.158875 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114e49b-59dc-46e9-83bf-ecbc0564bea5 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:00.159162 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114e49b-59dc-46e9-83bf-ecbc0564bea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:00.159432 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114e49b-59dc-46e9-83bf-ecbc0564bea5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:00.160187 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9114e49b-59dc-46e9-83bf-ecbc0564bea5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:26:00.160526 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9114e49b-59dc-46e9-83bf-ecbc0564bea5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:00.164859 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9114e49b-59dc-46e9-83bf-ecbc0564bea5 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:00.172830 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-b5590249-f67f-43c7-b267-2eac2d6d7466 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:00.175312 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-b5590249-f67f-43c7-b267-2eac2d6d7466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:00.185216 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 766/1554] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:26:00.185216 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174d917-349c-4494-9bbe-aaf3e9cdaef6 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:00.185216 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174d917-349c-4494-9bbe-aaf3e9cdaef6 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:00.185216 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174d917-349c-4494-9bbe-aaf3e9cdaef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:00.194936 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 789/1555] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:00.196479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50e3af-4224-47a6-8325-d0ce4a004fc7 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:00.196847 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50e3af-4224-47a6-8325-d0ce4a004fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:00.197261 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50e3af-4224-47a6-8325-d0ce4a004fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:00.211481 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-3174d917-349c-4494-9bbe-aaf3e9cdaef6 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:00.213045 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-3174d917-349c-4494-9bbe-aaf3e9cdaef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:00.218529 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-2b50e3af-4224-47a6-8325-d0ce4a004fc7 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:00.219773 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-2b50e3af-4224-47a6-8325-d0ce4a004fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:00.224237 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 767/1556] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:00.226520 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570cafb-35ec-4aea-8052-44799061eac6 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:00.226814 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570cafb-35ec-4aea-8052-44799061eac6 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:00.227253 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570cafb-35ec-4aea-8052-44799061eac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:00.228841 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 790/1557] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:00.230425 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4926f2-8beb-4d29-91ce-3d0f81c8aa06 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:00.230425 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4926f2-8beb-4d29-91ce-3d0f81c8aa06 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:00.230859 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4926f2-8beb-4d29-91ce-3d0f81c8aa06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:00.251467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-da4926f2-8beb-4d29-91ce-3d0f81c8aa06 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:00.252960 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-da4926f2-8beb-4d29-91ce-3d0f81c8aa06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:00.262047 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-8570cafb-35ec-4aea-8052-44799061eac6 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:00.268375 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 791/1558] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:00.268619 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8570cafb-35ec-4aea-8052-44799061eac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:00.268724 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4926f2-8beb-4d29-91ce-3d0f81c8aa06 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:00.272441 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 768/1559] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:00.273336 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570cafb-35ec-4aea-8052-44799061eac6 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:00.278846 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4926f2-8beb-4d29-91ce-3d0f81c8aa06 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:00.286963 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a061c42f-fcf1-4c88-9090-cc770e5c7dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:00.287814 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061c42f-fcf1-4c88-9090-cc770e5c7dae admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:00.288207 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061c42f-fcf1-4c88-9090-cc770e5c7dae admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:00.288614 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570cafb-35ec-4aea-8052-44799061eac6 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:00.288766 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061c42f-fcf1-4c88-9090-cc770e5c7dae admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:00.293421 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a061c42f-fcf1-4c88-9090-cc770e5c7dae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:00.293421 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a061c42f-fcf1-4c88-9090-cc770e5c7dae admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:00.293924 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23461388-e011-45f1-ba9a-9a37872ae13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:00.294620 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23461388-e011-45f1-ba9a-9a37872ae13c admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:00.294892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23461388-e011-45f1-ba9a-9a37872ae13c admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:00.295168 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23461388-e011-45f1-ba9a-9a37872ae13c admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:00.295912 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23461388-e011-45f1-ba9a-9a37872ae13c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:00.297540 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23461388-e011-45f1-ba9a-9a37872ae13c admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:00.303880 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 792/1560] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:00.305108 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a061c42f-fcf1-4c88-9090-cc770e5c7dae admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:00.308236 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 769/1561] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:00.309741 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d70b7-de11-43e8-b9fb-0ca31989fb6c None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:00.310132 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d70b7-de11-43e8-b9fb-0ca31989fb6c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:00.310480 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d70b7-de11-43e8-b9fb-0ca31989fb6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:00.321585 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a061c42f-fcf1-4c88-9090-cc770e5c7dae admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:00.326719 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5965b1f1-1cdf-4f96-93d3-d0d7e904b8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:00.328831 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965b1f1-1cdf-4f96-93d3-d0d7e904b8fb admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:00.329129 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965b1f1-1cdf-4f96-93d3-d0d7e904b8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:00.329496 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965b1f1-1cdf-4f96-93d3-d0d7e904b8fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:00.330519 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5965b1f1-1cdf-4f96-93d3-d0d7e904b8fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:00.334844 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5965b1f1-1cdf-4f96-93d3-d0d7e904b8fb admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:00.339976 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-534d70b7-de11-43e8-b9fb-0ca31989fb6c None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:00.341278 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-534d70b7-de11-43e8-b9fb-0ca31989fb6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:00.349950 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 770/1562] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:00.352881 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d70b7-de11-43e8-b9fb-0ca31989fb6c None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:00.363320 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d70b7-de11-43e8-b9fb-0ca31989fb6c None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:00.368761 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61d8861-062d-4063-978e-c6c2abf4870c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:00.374559 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d8861-062d-4063-978e-c6c2abf4870c admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:00.374559 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d8861-062d-4063-978e-c6c2abf4870c admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:00.374559 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d8861-062d-4063-978e-c6c2abf4870c admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:00.374559 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61d8861-062d-4063-978e-c6c2abf4870c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:26:00.374559 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61d8861-062d-4063-978e-c6c2abf4870c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:00.374559 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61d8861-062d-4063-978e-c6c2abf4870c admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:00.387511 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 771/1563] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:26:00.389215 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d0e41-4331-431c-bc48-9f5c58accf9a None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:00.389640 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d0e41-4331-431c-bc48-9f5c58accf9a None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:00.390040 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d0e41-4331-431c-bc48-9f5c58accf9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:00.409803 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-658d0e41-4331-431c-bc48-9f5c58accf9a None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:00.411051 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-658d0e41-4331-431c-bc48-9f5c58accf9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:00.419353 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 772/1564] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:00.420197 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d0e41-4331-431c-bc48-9f5c58accf9a None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:00.430380 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d0e41-4331-431c-bc48-9f5c58accf9a None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:00.435524 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae43a88-5cb9-4d05-8f62-8962ab517509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:00.437685 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae43a88-5cb9-4d05-8f62-8962ab517509 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:00.437685 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae43a88-5cb9-4d05-8f62-8962ab517509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:00.437685 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae43a88-5cb9-4d05-8f62-8962ab517509 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:00.437685 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae43a88-5cb9-4d05-8f62-8962ab517509 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:26:00.438320 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae43a88-5cb9-4d05-8f62-8962ab517509 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:00.439409 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae43a88-5cb9-4d05-8f62-8962ab517509 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:00.452310 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 773/1565] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:26:00.453703 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83f6e0-60bd-4968-9d1a-0411f3c61860 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:00.454221 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83f6e0-60bd-4968-9d1a-0411f3c61860 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:00.454352 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83f6e0-60bd-4968-9d1a-0411f3c61860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:00.469769 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 793/1566] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:26:00 2022] POST /identity/v3/projects => generated 373 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:00.469769 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5965b1f1-1cdf-4f96-93d3-d0d7e904b8fb admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:00.474047 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-fd83f6e0-60bd-4968-9d1a-0411f3c61860 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:00.475343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-fd83f6e0-60bd-4968-9d1a-0411f3c61860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:00.481681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:00.502143 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:00.535619 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 774/1567] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:00.536035 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5965b1f1-1cdf-4f96-93d3-d0d7e904b8fb admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:00.536541 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd83f6e0-60bd-4968-9d1a-0411f3c61860 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:00.543781 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ec217-9519-46b0-a112-0f9a638dbce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:00.544550 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ec217-9519-46b0-a112-0f9a638dbce3 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:00.544808 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ec217-9519-46b0-a112-0f9a638dbce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:00.545097 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ec217-9519-46b0-a112-0f9a638dbce3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:00.545838 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157ec217-9519-46b0-a112-0f9a638dbce3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:00.546321 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd83f6e0-60bd-4968-9d1a-0411f3c61860 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:00.547570 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157ec217-9519-46b0-a112-0f9a638dbce3 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:00.551425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35196470-259c-49e8-b230-3548c1bcb22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:00.552103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35196470-259c-49e8-b230-3548c1bcb22c admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:00.552467 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35196470-259c-49e8-b230-3548c1bcb22c admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:00.553274 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35196470-259c-49e8-b230-3548c1bcb22c admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:00.554202 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35196470-259c-49e8-b230-3548c1bcb22c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:00.555804 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35196470-259c-49e8-b230-3548c1bcb22c admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:00.558715 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 794/1568] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:00.559575 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ec217-9519-46b0-a112-0f9a638dbce3 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:00.566526 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 775/1569] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:00.567341 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35196470-259c-49e8-b230-3548c1bcb22c admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:00.573826 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ec217-9519-46b0-a112-0f9a638dbce3 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:00.578920 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d61dc-7595-4faa-8220-7b7c720f9853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:00.579591 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d61dc-7595-4faa-8220-7b7c720f9853 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:00.579873 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d61dc-7595-4faa-8220-7b7c720f9853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:00.580636 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d61dc-7595-4faa-8220-7b7c720f9853 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:00.580946 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5d61dc-7595-4faa-8220-7b7c720f9853 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:26:00.581068 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35196470-259c-49e8-b230-3548c1bcb22c admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:00.581256 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5d61dc-7595-4faa-8220-7b7c720f9853 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:00.582940 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5d61dc-7595-4faa-8220-7b7c720f9853 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:00.586018 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f002a33-1e79-4edd-8686-b1a3d3bf7014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:00.586708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f002a33-1e79-4edd-8686-b1a3d3bf7014 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:00.586995 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f002a33-1e79-4edd-8686-b1a3d3bf7014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:00.587278 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f002a33-1e79-4edd-8686-b1a3d3bf7014 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:00.588085 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f002a33-1e79-4edd-8686-b1a3d3bf7014 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:00.590017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f002a33-1e79-4edd-8686-b1a3d3bf7014 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:00.597056 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 795/1570] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:26:00.598085 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d61dc-7595-4faa-8220-7b7c720f9853 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:00.612318 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d61dc-7595-4faa-8220-7b7c720f9853 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:00.618925 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829afad2-103a-4369-b49b-0ba52c5fb9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:00.619607 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829afad2-103a-4369-b49b-0ba52c5fb9bf admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:00.619899 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829afad2-103a-4369-b49b-0ba52c5fb9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:00.620175 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829afad2-103a-4369-b49b-0ba52c5fb9bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:00.620958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829afad2-103a-4369-b49b-0ba52c5fb9bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:26:00.621266 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829afad2-103a-4369-b49b-0ba52c5fb9bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:00.623250 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829afad2-103a-4369-b49b-0ba52c5fb9bf admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:00.643935 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 796/1571] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:26:00.643935 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829afad2-103a-4369-b49b-0ba52c5fb9bf admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:00.647337 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829afad2-103a-4369-b49b-0ba52c5fb9bf admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:00.651960 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee12dd-c485-4170-9984-722d58fa8129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:00.652661 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee12dd-c485-4170-9984-722d58fa8129 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:00.652946 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee12dd-c485-4170-9984-722d58fa8129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:00.653224 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee12dd-c485-4170-9984-722d58fa8129 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:00.653949 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ee12dd-c485-4170-9984-722d58fa8129 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:00.655894 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ee12dd-c485-4170-9984-722d58fa8129 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:00.911229 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 797/1572] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:26:00 2022] POST /identity/v3/projects => generated 367 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:00.911476 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 776/1573] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:26:00 2022] POST /identity/v3/projects => generated 385 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:00.912932 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee12dd-c485-4170-9984-722d58fa8129 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:00.918497 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f002a33-1e79-4edd-8686-b1a3d3bf7014 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:00.989761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee12dd-c485-4170-9984-722d58fa8129 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:00.995988 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f4c87-e8d4-4e3c-a5e8-ca78aa40f714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:00.997380 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f4c87-e8d4-4e3c-a5e8-ca78aa40f714 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:00.997736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f4c87-e8d4-4e3c-a5e8-ca78aa40f714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:00.998100 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f4c87-e8d4-4e3c-a5e8-ca78aa40f714 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:00.998977 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f4c87-e8d4-4e3c-a5e8-ca78aa40f714 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:01.001469 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f4c87-e8d4-4e3c-a5e8-ca78aa40f714 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:01.015332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f002a33-1e79-4edd-8686-b1a3d3bf7014 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:01.023754 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca3067-9c9e-43b6-9be6-85107922cb79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:01.024961 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca3067-9c9e-43b6-9be6-85107922cb79 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:01.025481 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca3067-9c9e-43b6-9be6-85107922cb79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:01.025981 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca3067-9c9e-43b6-9be6-85107922cb79 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:01.027352 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca3067-9c9e-43b6-9be6-85107922cb79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:26:01.027915 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca3067-9c9e-43b6-9be6-85107922cb79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:01.031047 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca3067-9c9e-43b6-9be6-85107922cb79 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:01.046242 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 777/1574] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:26:01.047090 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca3067-9c9e-43b6-9be6-85107922cb79 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:01.057936 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca3067-9c9e-43b6-9be6-85107922cb79 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:01.062600 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c6d24a-6731-4a6d-a795-5d7b204b2d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:01.063509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6d24a-6731-4a6d-a795-5d7b204b2d42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:01.075942 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6d24a-6731-4a6d-a795-5d7b204b2d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:01.075942 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6d24a-6731-4a6d-a795-5d7b204b2d42 admin admin] PATH_INFO: `/v3/projects/2e45db85b4354b6ea087cd43addc110c/users/683c8a9abae1455bb2ec773b7608d03c/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:01.081701 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c6d24a-6731-4a6d-a795-5d7b204b2d42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e45db85b4354b6ea087cd43addc110c, user_id=683c8a9abae1455bb2ec773b7608d03c, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:01.082745 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: WARNING py.warnings [None req-f9c6d24a-6731-4a6d-a795-5d7b204b2d42 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 06:26:01.082745 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: warnings.warn(msg) Jul 18 06:26:01.082745 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: Jul 18 06:26:01.085598 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c6d24a-6731-4a6d-a795-5d7b204b2d42 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:01.116035 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 798/1575] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:26:00 2022] POST /identity/v3/users => generated 287 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:01.127489 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f4c87-e8d4-4e3c-a5e8-ca78aa40f714 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:01.128663 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f4c87-e8d4-4e3c-a5e8-ca78aa40f714 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:01.134569 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd9c3e-f9f5-46a8-979a-36305e837eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:01.135749 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd9c3e-f9f5-46a8-979a-36305e837eaa admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:01.136142 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd9c3e-f9f5-46a8-979a-36305e837eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:01.136609 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd9c3e-f9f5-46a8-979a-36305e837eaa admin admin] PATH_INFO: `/v3/projects/8d18d6a16fe2472e86c4453f87eddc05/users/788e2a26fe274360aed5498ef853e013/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:01.155712 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd9c3e-f9f5-46a8-979a-36305e837eaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d18d6a16fe2472e86c4453f87eddc05, user_id=788e2a26fe274360aed5498ef853e013, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:01.158442 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd9c3e-f9f5-46a8-979a-36305e837eaa admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:01.252940 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 778/1576] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:01 2022] PUT /identity/v3/projects/2e45db85b4354b6ea087cd43addc110c/users/683c8a9abae1455bb2ec773b7608d03c/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:01.253695 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c6d24a-6731-4a6d-a795-5d7b204b2d42 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:01.284997 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 799/1577] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:01 2022] PUT /identity/v3/projects/8d18d6a16fe2472e86c4453f87eddc05/users/788e2a26fe274360aed5498ef853e013/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:01.286775 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd9c3e-f9f5-46a8-979a-36305e837eaa admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:01.316749 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:01.330485 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c6d24a-6731-4a6d-a795-5d7b204b2d42 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:01.335996 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:01.356779 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd9c3e-f9f5-46a8-979a-36305e837eaa admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:01.363132 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71c0d9-0a72-4f52-a4fc-b6b3ce3cdd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:01.363994 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71c0d9-0a72-4f52-a4fc-b6b3ce3cdd28 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:01.364511 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71c0d9-0a72-4f52-a4fc-b6b3ce3cdd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:01.364965 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71c0d9-0a72-4f52-a4fc-b6b3ce3cdd28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:01.366766 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf71c0d9-0a72-4f52-a4fc-b6b3ce3cdd28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:01.368915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf71c0d9-0a72-4f52-a4fc-b6b3ce3cdd28 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:01.395417 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9f85a5-231e-44c6-891b-133c66e93518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:01.396085 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f85a5-231e-44c6-891b-133c66e93518 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:01.396397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f85a5-231e-44c6-891b-133c66e93518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:01.396711 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f85a5-231e-44c6-891b-133c66e93518 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:01.397436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9f85a5-231e-44c6-891b-133c66e93518 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:01.399460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9f85a5-231e-44c6-891b-133c66e93518 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:01.528312 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 800/1578] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:26:01 2022] POST /identity/v3/projects => generated 375 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:01.529307 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71c0d9-0a72-4f52-a4fc-b6b3ce3cdd28 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:01.564446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 779/1579] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:26:01 2022] POST /identity/v3/projects => generated 393 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:01.564446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9f85a5-231e-44c6-891b-133c66e93518 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:01.595332 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71c0d9-0a72-4f52-a4fc-b6b3ce3cdd28 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:01.638319 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9f85a5-231e-44c6-891b-133c66e93518 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:01.643231 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e631bc69-c6ed-4820-85de-46227878a1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:01.643925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631bc69-c6ed-4820-85de-46227878a1ab admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:01.644218 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631bc69-c6ed-4820-85de-46227878a1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:01.644528 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631bc69-c6ed-4820-85de-46227878a1ab admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:01.645317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e631bc69-c6ed-4820-85de-46227878a1ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:01.647363 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e631bc69-c6ed-4820-85de-46227878a1ab admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:01.670761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d4de89-e973-444f-a6ad-f992d7f76776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:01.671615 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4de89-e973-444f-a6ad-f992d7f76776 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:01.671899 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4de89-e973-444f-a6ad-f992d7f76776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:01.679117 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4de89-e973-444f-a6ad-f992d7f76776 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:01.679117 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d4de89-e973-444f-a6ad-f992d7f76776 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:01.679117 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d4de89-e973-444f-a6ad-f992d7f76776 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:01.790625 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 780/1580] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:26:01 2022] POST /identity/v3/users => generated 293 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:01.791545 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e631bc69-c6ed-4820-85de-46227878a1ab admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:01.801906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e631bc69-c6ed-4820-85de-46227878a1ab admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:01.807704 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a30a75-08db-456a-9272-57408aa5f36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:01.808375 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a30a75-08db-456a-9272-57408aa5f36d admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:01.808698 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a30a75-08db-456a-9272-57408aa5f36d admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:01.808980 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a30a75-08db-456a-9272-57408aa5f36d admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:01.809833 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a30a75-08db-456a-9272-57408aa5f36d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:01.813501 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 801/1581] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:26:01 2022] POST /identity/v3/users => generated 284 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:01.813670 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a30a75-08db-456a-9272-57408aa5f36d admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:01.813770 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d4de89-e973-444f-a6ad-f992d7f76776 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:01.823264 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d4de89-e973-444f-a6ad-f992d7f76776 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:01.828089 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cfa143-7c03-458d-aaf7-f1b0b17e7f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:01.828933 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cfa143-7c03-458d-aaf7-f1b0b17e7f1a admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:01.830967 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cfa143-7c03-458d-aaf7-f1b0b17e7f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:01.830967 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cfa143-7c03-458d-aaf7-f1b0b17e7f1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:01.830967 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cfa143-7c03-458d-aaf7-f1b0b17e7f1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:01.832311 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cfa143-7c03-458d-aaf7-f1b0b17e7f1a admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:01.844363 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 802/1582] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:01.850158 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f6ac3-fcc4-4b86-b50d-be16a51508e8 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:01.850158 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f6ac3-fcc4-4b86-b50d-be16a51508e8 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:01.850158 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f6ac3-fcc4-4b86-b50d-be16a51508e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:01.866327 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-4f1f6ac3-fcc4-4b86-b50d-be16a51508e8 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:01.867611 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4f1f6ac3-fcc4-4b86-b50d-be16a51508e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:01.875628 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 803/1583] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:01.877028 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3def7af-f55f-4c9a-ae9d-9b0aaed3f758 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:01.877320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3def7af-f55f-4c9a-ae9d-9b0aaed3f758 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:01.877600 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3def7af-f55f-4c9a-ae9d-9b0aaed3f758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:01.895329 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-c3def7af-f55f-4c9a-ae9d-9b0aaed3f758 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:01.896885 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c3def7af-f55f-4c9a-ae9d-9b0aaed3f758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:01.904773 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 804/1584] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:01.905671 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3def7af-f55f-4c9a-ae9d-9b0aaed3f758 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:01.917404 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3def7af-f55f-4c9a-ae9d-9b0aaed3f758 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:01.922661 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e33d3f-7233-486f-a7c4-aef62bc12e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:01.922975 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e33d3f-7233-486f-a7c4-aef62bc12e9b admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:01.923265 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e33d3f-7233-486f-a7c4-aef62bc12e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:01.923569 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e33d3f-7233-486f-a7c4-aef62bc12e9b admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:01.924264 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e33d3f-7233-486f-a7c4-aef62bc12e9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:01.926299 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e33d3f-7233-486f-a7c4-aef62bc12e9b admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:01.944939 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 781/1585] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:26:01 2022] POST /identity/v3/projects => generated 401 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:01.945898 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a30a75-08db-456a-9272-57408aa5f36d admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:01.999305 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:02.014485 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a30a75-08db-456a-9272-57408aa5f36d admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:02.019289 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d976608-83c7-4963-ab5f-64ba7d11a4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:02.020014 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d976608-83c7-4963-ab5f-64ba7d11a4ad admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:02.020297 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d976608-83c7-4963-ab5f-64ba7d11a4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:02.020636 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d976608-83c7-4963-ab5f-64ba7d11a4ad admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:02.021823 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d976608-83c7-4963-ab5f-64ba7d11a4ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:02.024611 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d976608-83c7-4963-ab5f-64ba7d11a4ad admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:02.035372 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 805/1586] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:26:01 2022] POST /identity/v3/users => generated 288 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:02.036367 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e33d3f-7233-486f-a7c4-aef62bc12e9b admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:02.049422 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e33d3f-7233-486f-a7c4-aef62bc12e9b admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:02.057481 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b0642-21c9-4edf-899a-4fcdf3fca191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:02.058652 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b0642-21c9-4edf-899a-4fcdf3fca191 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:02.059222 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b0642-21c9-4edf-899a-4fcdf3fca191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:02.059725 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b0642-21c9-4edf-899a-4fcdf3fca191 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:02.061184 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557b0642-21c9-4edf-899a-4fcdf3fca191 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:02.064432 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557b0642-21c9-4edf-899a-4fcdf3fca191 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:02.085895 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 806/1587] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:02.087450 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b0642-21c9-4edf-899a-4fcdf3fca191 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:02.105466 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b0642-21c9-4edf-899a-4fcdf3fca191 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:02.110999 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00192d4-5e98-453d-bff9-35549a1fe326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:02.111808 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00192d4-5e98-453d-bff9-35549a1fe326 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:02.112189 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00192d4-5e98-453d-bff9-35549a1fe326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:02.112608 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00192d4-5e98-453d-bff9-35549a1fe326 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:02.113559 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00192d4-5e98-453d-bff9-35549a1fe326 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:02.115316 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00192d4-5e98-453d-bff9-35549a1fe326 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:02.118219 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:02.131327 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 807/1588] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:02.131327 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00192d4-5e98-453d-bff9-35549a1fe326 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:02.139938 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00192d4-5e98-453d-bff9-35549a1fe326 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:02.144864 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a52d4b-ad24-4529-97dd-9a573d407069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:02.145563 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a52d4b-ad24-4529-97dd-9a573d407069 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:02.145851 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a52d4b-ad24-4529-97dd-9a573d407069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:02.146160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a52d4b-ad24-4529-97dd-9a573d407069 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:02.147568 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a52d4b-ad24-4529-97dd-9a573d407069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:02.149288 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a52d4b-ad24-4529-97dd-9a573d407069 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:02.160740 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 808/1589] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:02.161807 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a52d4b-ad24-4529-97dd-9a573d407069 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:02.184477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 782/1590] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:26:01 2022] POST /identity/v3/users => generated 297 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:02.185347 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d976608-83c7-4963-ab5f-64ba7d11a4ad admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:02.228894 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a52d4b-ad24-4529-97dd-9a573d407069 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:02.235109 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87eb954-b389-4309-b5c7-a5241b5f4e8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:02.235985 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87eb954-b389-4309-b5c7-a5241b5f4e8c service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:02.236520 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87eb954-b389-4309-b5c7-a5241b5f4e8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:02.236997 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87eb954-b389-4309-b5c7-a5241b5f4e8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:02.250999 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87eb954-b389-4309-b5c7-a5241b5f4e8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:02.253138 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87eb954-b389-4309-b5c7-a5241b5f4e8c service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:02.257403 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d976608-83c7-4963-ab5f-64ba7d11a4ad admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:02.261584 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d8f597-69dd-4f58-b034-dd163e488dd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:02.262371 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8f597-69dd-4f58-b034-dd163e488dd0 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:02.262659 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8f597-69dd-4f58-b034-dd163e488dd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:02.262939 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8f597-69dd-4f58-b034-dd163e488dd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:02.266167 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 809/1591] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:02.266849 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87eb954-b389-4309-b5c7-a5241b5f4e8c service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:02.274683 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d8f597-69dd-4f58-b034-dd163e488dd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:02.276256 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d8f597-69dd-4f58-b034-dd163e488dd0 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:02.279352 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87eb954-b389-4309-b5c7-a5241b5f4e8c service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:02.285408 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e491fbfb-0cf2-4a2c-98ea-44081b801567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:02.286509 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491fbfb-0cf2-4a2c-98ea-44081b801567 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:02.286509 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491fbfb-0cf2-4a2c-98ea-44081b801567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:02.286774 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491fbfb-0cf2-4a2c-98ea-44081b801567 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:02.288397 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e491fbfb-0cf2-4a2c-98ea-44081b801567 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:02.290163 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e491fbfb-0cf2-4a2c-98ea-44081b801567 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:02.291166 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 783/1592] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:02.292041 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d8f597-69dd-4f58-b034-dd163e488dd0 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:02.311730 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d8f597-69dd-4f58-b034-dd163e488dd0 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:02.319402 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb57919-bf87-4a78-9397-c7c6b687e4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:02.320730 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb57919-bf87-4a78-9397-c7c6b687e4b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:02.321317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb57919-bf87-4a78-9397-c7c6b687e4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:02.321814 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb57919-bf87-4a78-9397-c7c6b687e4b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:02.323200 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb57919-bf87-4a78-9397-c7c6b687e4b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:02.326360 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb57919-bf87-4a78-9397-c7c6b687e4b6 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:02.346165 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 784/1593] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:02.347888 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb57919-bf87-4a78-9397-c7c6b687e4b6 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:02.363467 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb57919-bf87-4a78-9397-c7c6b687e4b6 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:02.369077 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45909008-434b-4e36-875b-c24c6b67ce0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:02.369867 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45909008-434b-4e36-875b-c24c6b67ce0a admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:02.370230 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45909008-434b-4e36-875b-c24c6b67ce0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:02.370759 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45909008-434b-4e36-875b-c24c6b67ce0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:02.371563 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45909008-434b-4e36-875b-c24c6b67ce0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:02.374619 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45909008-434b-4e36-875b-c24c6b67ce0a admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:02.388996 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 785/1594] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:02.389878 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45909008-434b-4e36-875b-c24c6b67ce0a admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:02.401284 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45909008-434b-4e36-875b-c24c6b67ce0a admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:02.409443 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea80f211-3c03-46d9-a30e-6d81c02cba40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:02.410280 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80f211-3c03-46d9-a30e-6d81c02cba40 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:02.410580 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80f211-3c03-46d9-a30e-6d81c02cba40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:02.410856 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80f211-3c03-46d9-a30e-6d81c02cba40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:02.411637 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea80f211-3c03-46d9-a30e-6d81c02cba40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:02.414105 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea80f211-3c03-46d9-a30e-6d81c02cba40 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:02.419528 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 810/1595] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:26:02 2022] POST /identity/v3/users => generated 301 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:02.420464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e491fbfb-0cf2-4a2c-98ea-44081b801567 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:02.427949 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 786/1596] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:02.429092 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea80f211-3c03-46d9-a30e-6d81c02cba40 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:02.435386 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e491fbfb-0cf2-4a2c-98ea-44081b801567 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:02.441767 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd9ba7-5b07-4db5-9e07-fb185c31b432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:02.442843 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd9ba7-5b07-4db5-9e07-fb185c31b432 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:02.443132 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd9ba7-5b07-4db5-9e07-fb185c31b432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:02.444275 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd9ba7-5b07-4db5-9e07-fb185c31b432 admin admin] PATH_INFO: `/v3/projects/93582115dfa34cf494ca6fdd51b68357/users/693a660dbdb4442e874373978651f295/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:02.444566 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea80f211-3c03-46d9-a30e-6d81c02cba40 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:02.454734 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a861b9a-4f46-4543-9234-d339a1493f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:02.456768 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a861b9a-4f46-4543-9234-d339a1493f4f admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:02.460733 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a861b9a-4f46-4543-9234-d339a1493f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:02.460733 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a861b9a-4f46-4543-9234-d339a1493f4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:02.460733 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a861b9a-4f46-4543-9234-d339a1493f4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:02.463118 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a861b9a-4f46-4543-9234-d339a1493f4f admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:02.463817 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd9ba7-5b07-4db5-9e07-fb185c31b432 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93582115dfa34cf494ca6fdd51b68357, user_id=693a660dbdb4442e874373978651f295, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:02.466571 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd9ba7-5b07-4db5-9e07-fb185c31b432 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:02.485663 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 787/1597] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:02.486491 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a861b9a-4f46-4543-9234-d339a1493f4f admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:02.502869 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a861b9a-4f46-4543-9234-d339a1493f4f admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:02.508732 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92f6f6-1bd1-42f9-af4d-e085d4e3c666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:02.509416 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92f6f6-1bd1-42f9-af4d-e085d4e3c666 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:02.509705 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92f6f6-1bd1-42f9-af4d-e085d4e3c666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:02.509983 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92f6f6-1bd1-42f9-af4d-e085d4e3c666 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:02.510724 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac92f6f6-1bd1-42f9-af4d-e085d4e3c666 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:02.512368 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac92f6f6-1bd1-42f9-af4d-e085d4e3c666 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:02.525456 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 788/1598] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:02.526464 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92f6f6-1bd1-42f9-af4d-e085d4e3c666 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:02.537729 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92f6f6-1bd1-42f9-af4d-e085d4e3c666 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:02.555063 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 811/1599] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:02 2022] PUT /identity/v3/projects/93582115dfa34cf494ca6fdd51b68357/users/693a660dbdb4442e874373978651f295/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:02.556815 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd9ba7-5b07-4db5-9e07-fb185c31b432 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:02.623183 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f27f9-ff37-45cd-9c5b-c6aed441af2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:02.624824 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f27f9-ff37-45cd-9c5b-c6aed441af2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:02.625126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f27f9-ff37-45cd-9c5b-c6aed441af2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:02.625413 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f27f9-ff37-45cd-9c5b-c6aed441af2a admin admin] PATH_INFO: `/v3/projects/404c5f366b1c49bab19fbf26cadf7029/users/9315aedb45514abfa86265acfc5eae30/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:02.643671 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f27f9-ff37-45cd-9c5b-c6aed441af2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=404c5f366b1c49bab19fbf26cadf7029, user_id=9315aedb45514abfa86265acfc5eae30, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:02.647054 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f27f9-ff37-45cd-9c5b-c6aed441af2a admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:02.652688 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd9ba7-5b07-4db5-9e07-fb185c31b432 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:02.657226 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a73ab0-a3e9-4e9b-8fa4-55cf6e7487fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:02.657916 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a73ab0-a3e9-4e9b-8fa4-55cf6e7487fd admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:02.658239 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a73ab0-a3e9-4e9b-8fa4-55cf6e7487fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:02.658540 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a73ab0-a3e9-4e9b-8fa4-55cf6e7487fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:02.659246 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a73ab0-a3e9-4e9b-8fa4-55cf6e7487fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:02.660897 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a73ab0-a3e9-4e9b-8fa4-55cf6e7487fd admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:02.672149 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 812/1600] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:02.672999 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a73ab0-a3e9-4e9b-8fa4-55cf6e7487fd admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:02.683279 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a73ab0-a3e9-4e9b-8fa4-55cf6e7487fd admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:02.688203 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f9b472-ea85-47d9-a742-d0f994f8f1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:02.689149 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9b472-ea85-47d9-a742-d0f994f8f1f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:02.689441 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9b472-ea85-47d9-a742-d0f994f8f1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:02.690179 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9b472-ea85-47d9-a742-d0f994f8f1f3 admin admin] PATH_INFO: `/v3/projects/56f4eb50942b4e2eba5c3db4149a1e79/users/d35fb3e36d7a46a0ab79e23eb2286bb7/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:02.706630 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f9b472-ea85-47d9-a742-d0f994f8f1f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56f4eb50942b4e2eba5c3db4149a1e79, user_id=d35fb3e36d7a46a0ab79e23eb2286bb7, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:02.710003 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f9b472-ea85-47d9-a742-d0f994f8f1f3 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:02.852477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 789/1601] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:02 2022] PUT /identity/v3/projects/404c5f366b1c49bab19fbf26cadf7029/users/9315aedb45514abfa86265acfc5eae30/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:02.853512 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f27f9-ff37-45cd-9c5b-c6aed441af2a admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:02.886404 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:02.907347 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 813/1602] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:02 2022] PUT /identity/v3/projects/56f4eb50942b4e2eba5c3db4149a1e79/users/d35fb3e36d7a46a0ab79e23eb2286bb7/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:02.928748 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f9b472-ea85-47d9-a742-d0f994f8f1f3 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:02.952933 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:02.976912 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f27f9-ff37-45cd-9c5b-c6aed441af2a admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:03.024464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f9b472-ea85-47d9-a742-d0f994f8f1f3 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:03.032986 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1626e7-de4c-41d5-ac63-23fa99058a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:03.034640 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1626e7-de4c-41d5-ac63-23fa99058a59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:03.035175 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1626e7-de4c-41d5-ac63-23fa99058a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:03.035693 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1626e7-de4c-41d5-ac63-23fa99058a59 admin admin] PATH_INFO: `/v3/projects/da1cc5ce226a40a7ab6a90e08f43a0e4/users/11e2d1981b984919903cbbba65963c91/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:03.042447 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5709c82-bc65-4693-83b8-040728de78a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:03.043408 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5709c82-bc65-4693-83b8-040728de78a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:03.043833 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5709c82-bc65-4693-83b8-040728de78a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:03.044264 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5709c82-bc65-4693-83b8-040728de78a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:03.045355 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5709c82-bc65-4693-83b8-040728de78a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:03.047633 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5709c82-bc65-4693-83b8-040728de78a3 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:03.061433 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1626e7-de4c-41d5-ac63-23fa99058a59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da1cc5ce226a40a7ab6a90e08f43a0e4, user_id=11e2d1981b984919903cbbba65963c91, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:03.062472 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 790/1603] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:03.064479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1626e7-de4c-41d5-ac63-23fa99058a59 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:03.085525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992b928-05f1-48f9-92a8-4b4e2bcbb336 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:03.085525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992b928-05f1-48f9-92a8-4b4e2bcbb336 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:03.085525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992b928-05f1-48f9-92a8-4b4e2bcbb336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:03.098172 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-6992b928-05f1-48f9-92a8-4b4e2bcbb336 None None] MFA Rules not processed for user `693a660dbdb4442e874373978651f295`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:03.100444 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-6992b928-05f1-48f9-92a8-4b4e2bcbb336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:03.208312 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 791/1604] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:03.209228 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992b928-05f1-48f9-92a8-4b4e2bcbb336 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:03.228198 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 814/1605] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:02 2022] PUT /identity/v3/projects/da1cc5ce226a40a7ab6a90e08f43a0e4/users/11e2d1981b984919903cbbba65963c91/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:03.230842 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dabc21-407a-4ba5-a601-018a7fa432c8 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:03.231423 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dabc21-407a-4ba5-a601-018a7fa432c8 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:03.231558 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dabc21-407a-4ba5-a601-018a7fa432c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:03.257109 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992b928-05f1-48f9-92a8-4b4e2bcbb336 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:03.316304 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:03.317136 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-76dabc21-407a-4ba5-a601-018a7fa432c8 None None] MFA Rules not processed for user `9315aedb45514abfa86265acfc5eae30`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:03.322748 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-76dabc21-407a-4ba5-a601-018a7fa432c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:03.337422 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b6f9ce-13c6-427e-a7b6-d459a3b951ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:03.338108 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6f9ce-13c6-427e-a7b6-d459a3b951ac admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:03.338398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6f9ce-13c6-427e-a7b6-d459a3b951ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:03.338674 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6f9ce-13c6-427e-a7b6-d459a3b951ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:03.339414 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b6f9ce-13c6-427e-a7b6-d459a3b951ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:03.341282 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b6f9ce-13c6-427e-a7b6-d459a3b951ac admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:03.358688 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 792/1606] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:03.358688 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6b776-c20d-4db8-b403-5a7c10680ee4 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:03.358688 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6b776-c20d-4db8-b403-5a7c10680ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:03.358688 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6b776-c20d-4db8-b403-5a7c10680ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:03.392663 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-06c6b776-c20d-4db8-b403-5a7c10680ee4 None None] MFA Rules not processed for user `d35fb3e36d7a46a0ab79e23eb2286bb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:03.393885 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-06c6b776-c20d-4db8-b403-5a7c10680ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:03.451788 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 815/1607] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:03.458224 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dabc21-407a-4ba5-a601-018a7fa432c8 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:03.470981 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dabc21-407a-4ba5-a601-018a7fa432c8 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:03.477470 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0780909-4cbc-4f52-9fe0-f181a7e655dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:03.478546 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0780909-4cbc-4f52-9fe0-f181a7e655dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:03.478924 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0780909-4cbc-4f52-9fe0-f181a7e655dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:03.479246 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0780909-4cbc-4f52-9fe0-f181a7e655dd admin admin] PATH_INFO: `/v3/projects/c8f4f0c21b95422f80245b76b56367fb/users/76f3dd56c5d24c7490e337b2aa90b584/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:03.496014 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 793/1608] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:03 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:03.499134 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0780909-4cbc-4f52-9fe0-f181a7e655dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8f4f0c21b95422f80245b76b56367fb, user_id=76f3dd56c5d24c7490e337b2aa90b584, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:03.500943 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56be39-56f6-45a4-87a4-b9e8e2eb71bb None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:03.501835 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56be39-56f6-45a4-87a4-b9e8e2eb71bb None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:03.502323 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0780909-4cbc-4f52-9fe0-f181a7e655dd admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:03.502952 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56be39-56f6-45a4-87a4-b9e8e2eb71bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:03.551802 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-7a56be39-56f6-45a4-87a4-b9e8e2eb71bb None None] MFA Rules not processed for user `11e2d1981b984919903cbbba65963c91`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:03.555607 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7a56be39-56f6-45a4-87a4-b9e8e2eb71bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:03.662510 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 816/1609] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:03 2022] PUT /identity/v3/projects/c8f4f0c21b95422f80245b76b56367fb/users/76f3dd56c5d24c7490e337b2aa90b584/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:03.665822 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fb21a-a139-4d96-bcb7-b4c36adc4164 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:03.666348 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fb21a-a139-4d96-bcb7-b4c36adc4164 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:03.667264 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fb21a-a139-4d96-bcb7-b4c36adc4164 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:03.669990 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 817/1610] 10.222.0.40 () {54 vars in 962 bytes} [Mon Jul 18 06:26:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:26:03.671349 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7fb21a-a139-4d96-bcb7-b4c36adc4164 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:03.673061 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 794/1611] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:03.673953 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a56be39-56f6-45a4-87a4-b9e8e2eb71bb None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:03.690987 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a56be39-56f6-45a4-87a4-b9e8e2eb71bb None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:03.766194 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a073b5b-a901-4362-be72-3b0d1a061da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:03.766994 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a073b5b-a901-4362-be72-3b0d1a061da4 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:03.767253 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a073b5b-a901-4362-be72-3b0d1a061da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:03.767560 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a073b5b-a901-4362-be72-3b0d1a061da4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:03.768270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a073b5b-a901-4362-be72-3b0d1a061da4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:03.770563 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a073b5b-a901-4362-be72-3b0d1a061da4 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:03.779707 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7fb21a-a139-4d96-bcb7-b4c36adc4164 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:03.784627 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06744aa-69ff-4f66-884a-bc27afafe732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:03.785583 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06744aa-69ff-4f66-884a-bc27afafe732 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:03.785875 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06744aa-69ff-4f66-884a-bc27afafe732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:03.786204 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06744aa-69ff-4f66-884a-bc27afafe732 admin admin] PATH_INFO: `/v3/projects/0f5c0c9873314a8baff2d32bf64c162a/users/79ac4b77d2704391b0512d9c80995342/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:03.800122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:03.803882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06744aa-69ff-4f66-884a-bc27afafe732 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f5c0c9873314a8baff2d32bf64c162a, user_id=79ac4b77d2704391b0512d9c80995342, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:03.806698 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06744aa-69ff-4f66-884a-bc27afafe732 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:03.914899 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 795/1612] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:26:03 2022] POST /identity/v3/projects => generated 387 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:03.915855 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a073b5b-a901-4362-be72-3b0d1a061da4 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:03.966882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 818/1613] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:03 2022] PUT /identity/v3/projects/0f5c0c9873314a8baff2d32bf64c162a/users/79ac4b77d2704391b0512d9c80995342/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:03.968305 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2fc319-c216-4ad6-8d5e-7a72bfff0de8 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:03.968783 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2fc319-c216-4ad6-8d5e-7a72bfff0de8 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:03.969081 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2fc319-c216-4ad6-8d5e-7a72bfff0de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:03.972986 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a073b5b-a901-4362-be72-3b0d1a061da4 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:03.987587 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-cf2fc319-c216-4ad6-8d5e-7a72bfff0de8 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:03.988994 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-cf2fc319-c216-4ad6-8d5e-7a72bfff0de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:04.032148 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084db2d-d3cf-463f-a954-4e84c84e7e5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:04.034751 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084db2d-d3cf-463f-a954-4e84c84e7e5c service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:04.034751 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084db2d-d3cf-463f-a954-4e84c84e7e5c service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:04.034751 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084db2d-d3cf-463f-a954-4e84c84e7e5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:04.063325 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 819/1614] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:04.063325 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb213ec-a761-4150-ab31-a55e4cfef604 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:04.063325 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb213ec-a761-4150-ab31-a55e4cfef604 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:04.063325 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb213ec-a761-4150-ab31-a55e4cfef604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:04.098970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:04.107210 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-efb213ec-a761-4150-ab31-a55e4cfef604 None None] MFA Rules not processed for user `f92d948810ea4241810473506b654f7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:04.108488 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-efb213ec-a761-4150-ab31-a55e4cfef604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:04.113459 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4084db2d-d3cf-463f-a954-4e84c84e7e5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:04.114946 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4084db2d-d3cf-463f-a954-4e84c84e7e5c service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:04.126735 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 796/1615] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:03 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:04.128250 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f02f15-714f-4e08-b9d6-8dafdd3faac1 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:04.128581 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f02f15-714f-4e08-b9d6-8dafdd3faac1 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:04.128942 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f02f15-714f-4e08-b9d6-8dafdd3faac1 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:04.130536 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 797/1616] 10.222.0.40 () {54 vars in 962 bytes} [Mon Jul 18 06:26:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:26:04.131502 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f02f15-714f-4e08-b9d6-8dafdd3faac1 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:04.149438 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f02f15-714f-4e08-b9d6-8dafdd3faac1 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:04.155585 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c2383-fc23-4e68-bd37-8794a7e86577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:04.156467 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c2383-fc23-4e68-bd37-8794a7e86577 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:04.156755 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c2383-fc23-4e68-bd37-8794a7e86577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:04.157038 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c2383-fc23-4e68-bd37-8794a7e86577 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:04.158138 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c2383-fc23-4e68-bd37-8794a7e86577 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:04.161514 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c2383-fc23-4e68-bd37-8794a7e86577 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:04.205617 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 820/1617] 10.222.0.40 () {58 vars in 1059 bytes} [Mon Jul 18 06:26:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:04.211113 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0159e-18d6-467f-ae52-7753b1a4b461 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:04.211446 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0159e-18d6-467f-ae52-7753b1a4b461 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:04.211737 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0159e-18d6-467f-ae52-7753b1a4b461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:04.245416 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-9cb0159e-18d6-467f-ae52-7753b1a4b461 None None] MFA Rules not processed for user `79ac4b77d2704391b0512d9c80995342`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:04.246690 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-9cb0159e-18d6-467f-ae52-7753b1a4b461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:04.308322 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 798/1618] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:26:04 2022] POST /identity/v3/users => generated 300 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:04.315419 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c2383-fc23-4e68-bd37-8794a7e86577 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:04.379997 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 821/1619] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:04 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:04.379997 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed2d16-b787-4e6c-bcca-2a88b8289733 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:04.379997 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed2d16-b787-4e6c-bcca-2a88b8289733 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:04.379997 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed2d16-b787-4e6c-bcca-2a88b8289733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:04.413541 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-71ed2d16-b787-4e6c-bcca-2a88b8289733 None None] MFA Rules not processed for user `f92d948810ea4241810473506b654f7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:04.415903 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-71ed2d16-b787-4e6c-bcca-2a88b8289733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:04.445022 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c2383-fc23-4e68-bd37-8794a7e86577 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:04.450240 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48def290-488b-4dd3-9a1f-1a6246b34b74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:04.450524 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48def290-488b-4dd3-9a1f-1a6246b34b74 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:04.450703 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48def290-488b-4dd3-9a1f-1a6246b34b74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:04.450982 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48def290-488b-4dd3-9a1f-1a6246b34b74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:04.458111 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 822/1620] 10.222.0.40 () {58 vars in 1059 bytes} [Mon Jul 18 06:26:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:04.460464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed2d16-b787-4e6c-bcca-2a88b8289733 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:04.465007 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48def290-488b-4dd3-9a1f-1a6246b34b74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:04.465410 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48def290-488b-4dd3-9a1f-1a6246b34b74 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:04.498193 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 799/1621] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:04.510872 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48def290-488b-4dd3-9a1f-1a6246b34b74 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:04.514421 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed2d16-b787-4e6c-bcca-2a88b8289733 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:04.529219 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f833f35-2482-4433-a9e5-210c218d0aa3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:04.529581 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48def290-488b-4dd3-9a1f-1a6246b34b74 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:04.529896 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f833f35-2482-4433-a9e5-210c218d0aa3 service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:04.530187 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f833f35-2482-4433-a9e5-210c218d0aa3 service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:04.530480 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f833f35-2482-4433-a9e5-210c218d0aa3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:04.541770 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fdbd8-861f-414b-9a59-a1a96a52f36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:04.543860 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fdbd8-861f-414b-9a59-a1a96a52f36a admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:04.543860 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fdbd8-861f-414b-9a59-a1a96a52f36a admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:04.544782 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fdbd8-861f-414b-9a59-a1a96a52f36a admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:04.547849 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216fdbd8-861f-414b-9a59-a1a96a52f36a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:04.557464 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216fdbd8-861f-414b-9a59-a1a96a52f36a admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:04.593511 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 800/1622] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:04.593511 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fdbd8-861f-414b-9a59-a1a96a52f36a admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:04.616511 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fdbd8-861f-414b-9a59-a1a96a52f36a admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:04.632840 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d3a5fc-512e-4ec1-9b30-a92fa0d95b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:04.635008 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3a5fc-512e-4ec1-9b30-a92fa0d95b92 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:04.635509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3a5fc-512e-4ec1-9b30-a92fa0d95b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:04.636027 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3a5fc-512e-4ec1-9b30-a92fa0d95b92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:04.638281 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d3a5fc-512e-4ec1-9b30-a92fa0d95b92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:04.643000 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d3a5fc-512e-4ec1-9b30-a92fa0d95b92 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:04.739791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f833f35-2482-4433-a9e5-210c218d0aa3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:04.744298 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f833f35-2482-4433-a9e5-210c218d0aa3 service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:04.773373 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 823/1623] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:04.775322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f833f35-2482-4433-a9e5-210c218d0aa3 service nova] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:04.799325 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f833f35-2482-4433-a9e5-210c218d0aa3 service nova] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:04.812074 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cb7f4d-3698-43a0-934f-8a127a022d9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:04.813472 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb7f4d-3698-43a0-934f-8a127a022d9a service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:04.814370 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb7f4d-3698-43a0-934f-8a127a022d9a service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:04.815097 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb7f4d-3698-43a0-934f-8a127a022d9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:04.857703 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 801/1624] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:26:04 2022] POST /identity/v3/projects => generated 403 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:04.860503 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d3a5fc-512e-4ec1-9b30-a92fa0d95b92 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:04.982239 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cb7f4d-3698-43a0-934f-8a127a022d9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:04.985574 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cb7f4d-3698-43a0-934f-8a127a022d9a service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:05.008993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 824/1625] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:05.010417 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cb7f4d-3698-43a0-934f-8a127a022d9a service nova] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:05.034370 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cb7f4d-3698-43a0-934f-8a127a022d9a service nova] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:05.127384 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:05.132578 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d3a5fc-512e-4ec1-9b30-a92fa0d95b92 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:05.154161 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e5da7-f75f-4a61-9dd8-3295085ff98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:05.155272 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e5da7-f75f-4a61-9dd8-3295085ff98e admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:05.155666 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e5da7-f75f-4a61-9dd8-3295085ff98e admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:05.156046 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e5da7-f75f-4a61-9dd8-3295085ff98e admin admin] PATH_INFO: `/v3/projects/32450e66e64c449288866dd4cb52f939/users/8c58dfcd728b4636a539c630f864bccd/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:05.210732 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e5da7-f75f-4a61-9dd8-3295085ff98e admin admin] RBAC: Authorizing `identity:create_grant(project_id=32450e66e64c449288866dd4cb52f939, user_id=8c58dfcd728b4636a539c630f864bccd, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:05.213747 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e5da7-f75f-4a61-9dd8-3295085ff98e admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:05.236262 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eba2a5-a0c3-4e6d-aecc-45b5b66e8c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:05.239114 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eba2a5-a0c3-4e6d-aecc-45b5b66e8c7e admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:05.239660 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eba2a5-a0c3-4e6d-aecc-45b5b66e8c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:05.240833 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eba2a5-a0c3-4e6d-aecc-45b5b66e8c7e admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:05.241851 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eba2a5-a0c3-4e6d-aecc-45b5b66e8c7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:05.248111 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eba2a5-a0c3-4e6d-aecc-45b5b66e8c7e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:05.356465 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 802/1626] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:04 2022] PUT /identity/v3/projects/32450e66e64c449288866dd4cb52f939/users/8c58dfcd728b4636a539c630f864bccd/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:05.360303 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e5da7-f75f-4a61-9dd8-3295085ff98e admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:05.428732 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 825/1627] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:26:05 2022] POST /identity/v3/users => generated 308 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:05.428732 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b0a9a-9e59-4616-8858-b32a177bcf44 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:05.428732 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b0a9a-9e59-4616-8858-b32a177bcf44 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:05.428732 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b0a9a-9e59-4616-8858-b32a177bcf44 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:05.440343 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 826/1628] 10.222.0.40 () {54 vars in 912 bytes} [Mon Jul 18 06:26:05 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:26:05.441267 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b0a9a-9e59-4616-8858-b32a177bcf44 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:05.491519 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e5da7-f75f-4a61-9dd8-3295085ff98e admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:05.502294 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe5658-470d-4f59-b036-1fb5691ff55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:05.505175 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe5658-470d-4f59-b036-1fb5691ff55a admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:05.506906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe5658-470d-4f59-b036-1fb5691ff55a admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:05.508093 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe5658-470d-4f59-b036-1fb5691ff55a admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:05.510328 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe5658-470d-4f59-b036-1fb5691ff55a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:05.512938 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe5658-470d-4f59-b036-1fb5691ff55a admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:05.571171 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 803/1629] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:05.574828 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5df9a-f1cb-4573-a0ea-1c990e93975f None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:05.575298 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5df9a-f1cb-4573-a0ea-1c990e93975f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:05.575455 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5df9a-f1cb-4573-a0ea-1c990e93975f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:05.611770 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b0a9a-9e59-4616-8858-b32a177bcf44 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:05.617994 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-24d5df9a-f1cb-4573-a0ea-1c990e93975f None None] MFA Rules not processed for user `f92d948810ea4241810473506b654f7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:05.619240 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-24d5df9a-f1cb-4573-a0ea-1c990e93975f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:05.621769 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a509a55-0362-4eba-956b-680a747a19ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:05.623822 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a509a55-0362-4eba-956b-680a747a19ef admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:05.624895 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a509a55-0362-4eba-956b-680a747a19ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:05.626851 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a509a55-0362-4eba-956b-680a747a19ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:05.628084 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a509a55-0362-4eba-956b-680a747a19ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:05.633070 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a509a55-0362-4eba-956b-680a747a19ef admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:05.643439 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:05.663788 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 827/1630] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:05.665315 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a509a55-0362-4eba-956b-680a747a19ef admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:05.684050 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a509a55-0362-4eba-956b-680a747a19ef admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:05.696874 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b42083-7dc5-46a6-b31d-d6afa97f7770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:05.697801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b42083-7dc5-46a6-b31d-d6afa97f7770 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:05.698129 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b42083-7dc5-46a6-b31d-d6afa97f7770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:05.698416 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b42083-7dc5-46a6-b31d-d6afa97f7770 admin admin] PATH_INFO: `/v3/projects/32450e66e64c449288866dd4cb52f939/users/8c58dfcd728b4636a539c630f864bccd/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:05.705601 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b42083-7dc5-46a6-b31d-d6afa97f7770 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32450e66e64c449288866dd4cb52f939, user_id=8c58dfcd728b4636a539c630f864bccd, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:05.710887 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b42083-7dc5-46a6-b31d-d6afa97f7770 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:05.776126 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 828/1631] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:05 2022] PUT /identity/v3/projects/32450e66e64c449288866dd4cb52f939/users/8c58dfcd728b4636a539c630f864bccd/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:05.776126 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859dbea-f559-4f19-85ea-1637862d66df None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:05.776126 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859dbea-f559-4f19-85ea-1637862d66df None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:05.776126 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859dbea-f559-4f19-85ea-1637862d66df None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:05.787171 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 829/1632] 10.222.0.40 () {54 vars in 912 bytes} [Mon Jul 18 06:26:05 2022] GET /identity => generated 270 bytes in 27 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:26:05.789700 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859dbea-f559-4f19-85ea-1637862d66df None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:05.830145 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 804/1633] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Jul 18 06:26:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:05.835326 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df633c-753e-4363-b4da-c5d0b762e2b1 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:05.836330 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df633c-753e-4363-b4da-c5d0b762e2b1 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:05.839518 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df633c-753e-4363-b4da-c5d0b762e2b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:05.944046 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-f4df633c-753e-4363-b4da-c5d0b762e2b1 None None] MFA Rules not processed for user `8c58dfcd728b4636a539c630f864bccd`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:05.946148 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f4df633c-753e-4363-b4da-c5d0b762e2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:05.993700 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859dbea-f559-4f19-85ea-1637862d66df None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:06.014909 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8868ca5-3550-4c10-b9ac-e7d1714559d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:06.017743 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8868ca5-3550-4c10-b9ac-e7d1714559d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:06.018207 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8868ca5-3550-4c10-b9ac-e7d1714559d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:06.018669 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8868ca5-3550-4c10-b9ac-e7d1714559d3 admin admin] PATH_INFO: `/v3/projects/8b1da02b710246e48ee8f8a64b15b45f/users/f0380a338e824375bab94af7cded3ea6/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:06.077951 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8868ca5-3550-4c10-b9ac-e7d1714559d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b1da02b710246e48ee8f8a64b15b45f, user_id=f0380a338e824375bab94af7cded3ea6, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:06.086916 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8868ca5-3550-4c10-b9ac-e7d1714559d3 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:06.119446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 805/1634] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:05 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:06.122356 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92323fe6-1bfb-43b0-8b3a-e16210d68910 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:06.123930 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92323fe6-1bfb-43b0-8b3a-e16210d68910 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:06.124533 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92323fe6-1bfb-43b0-8b3a-e16210d68910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:06.152022 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-92323fe6-1bfb-43b0-8b3a-e16210d68910 None None] MFA Rules not processed for user `f92d948810ea4241810473506b654f7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:06.154721 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-92323fe6-1bfb-43b0-8b3a-e16210d68910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:06.232930 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 830/1635] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:05 2022] PUT /identity/v3/projects/8b1da02b710246e48ee8f8a64b15b45f/users/f0380a338e824375bab94af7cded3ea6/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:06.234419 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8868ca5-3550-4c10-b9ac-e7d1714559d3 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:06.306385 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 806/1636] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Jul 18 06:26:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:06.308225 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92323fe6-1bfb-43b0-8b3a-e16210d68910 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:06.374363 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8868ca5-3550-4c10-b9ac-e7d1714559d3 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:06.381294 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dc55d6-7382-49cb-abaa-f0ff271076c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:06.382918 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc55d6-7382-49cb-abaa-f0ff271076c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:06.383641 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc55d6-7382-49cb-abaa-f0ff271076c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:06.384843 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc55d6-7382-49cb-abaa-f0ff271076c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:06.397435 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92323fe6-1bfb-43b0-8b3a-e16210d68910 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:06.402804 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053ebc7c-e2db-47b5-a2c9-f681cd4acc25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:06.403798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ebc7c-e2db-47b5-a2c9-f681cd4acc25 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:06.404397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ebc7c-e2db-47b5-a2c9-f681cd4acc25 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:06.404725 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ebc7c-e2db-47b5-a2c9-f681cd4acc25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:06.477108 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:06.543765 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053ebc7c-e2db-47b5-a2c9-f681cd4acc25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:06.546461 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053ebc7c-e2db-47b5-a2c9-f681cd4acc25 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:06.838268 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 807/1637] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:06 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:06.838268 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053ebc7c-e2db-47b5-a2c9-f681cd4acc25 service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:06.841121 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dc55d6-7382-49cb-abaa-f0ff271076c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:06.841121 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dc55d6-7382-49cb-abaa-f0ff271076c6 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:06.841121 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 831/1638] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:06.841121 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:06.841121 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dc55d6-7382-49cb-abaa-f0ff271076c6 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:06.841121 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dc55d6-7382-49cb-abaa-f0ff271076c6 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:06.845729 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053ebc7c-e2db-47b5-a2c9-f681cd4acc25 service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:06.846480 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62730b6-ee23-4d18-8165-cc2c0dd3c751 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:06.846480 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62730b6-ee23-4d18-8165-cc2c0dd3c751 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:06.846480 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62730b6-ee23-4d18-8165-cc2c0dd3c751 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:06.846480 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62730b6-ee23-4d18-8165-cc2c0dd3c751 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:06.847143 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddeac34-8b56-4386-9069-dac447dc355b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:06.847143 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddeac34-8b56-4386-9069-dac447dc355b admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:06.847143 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddeac34-8b56-4386-9069-dac447dc355b admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:06.847143 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddeac34-8b56-4386-9069-dac447dc355b admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:06.847747 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddeac34-8b56-4386-9069-dac447dc355b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:06.847747 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddeac34-8b56-4386-9069-dac447dc355b admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:06.847988 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62730b6-ee23-4d18-8165-cc2c0dd3c751 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:06.847988 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62730b6-ee23-4d18-8165-cc2c0dd3c751 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:06.848240 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 808/1639] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:06.848240 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddeac34-8b56-4386-9069-dac447dc355b admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:06.872907 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddeac34-8b56-4386-9069-dac447dc355b admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:06.875291 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 832/1640] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 06:26:06.883020 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3cfa09-3851-4891-9d51-491a7e41ec85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:06.885028 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3cfa09-3851-4891-9d51-491a7e41ec85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:06.885421 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3cfa09-3851-4891-9d51-491a7e41ec85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:06.885805 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3cfa09-3851-4891-9d51-491a7e41ec85 admin admin] PATH_INFO: `/v3/projects/8b1da02b710246e48ee8f8a64b15b45f/users/f0380a338e824375bab94af7cded3ea6/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:06.888649 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3cfa09-3851-4891-9d51-491a7e41ec85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b1da02b710246e48ee8f8a64b15b45f, user_id=f0380a338e824375bab94af7cded3ea6, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:06.893233 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3cfa09-3851-4891-9d51-491a7e41ec85 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:06.910616 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 809/1641] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:06 2022] PUT /identity/v3/projects/8b1da02b710246e48ee8f8a64b15b45f/users/f0380a338e824375bab94af7cded3ea6/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:06.929986 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61e3f0-4985-450c-9aee-dec612228ed9 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:06.930393 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61e3f0-4985-450c-9aee-dec612228ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:06.930590 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61e3f0-4985-450c-9aee-dec612228ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:06.972453 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-be61e3f0-4985-450c-9aee-dec612228ed9 None None] MFA Rules not processed for user `f0380a338e824375bab94af7cded3ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:06.976629 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-be61e3f0-4985-450c-9aee-dec612228ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:07.047163 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3cfa09-3851-4891-9d51-491a7e41ec85 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:07.108972 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:07.180940 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:07.249148 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3cfa09-3851-4891-9d51-491a7e41ec85 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:07.267388 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63516dd-4e87-4c1b-93d7-bcd474764d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:07.268766 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63516dd-4e87-4c1b-93d7-bcd474764d83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:07.269268 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63516dd-4e87-4c1b-93d7-bcd474764d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:07.269730 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63516dd-4e87-4c1b-93d7-bcd474764d83 admin admin] PATH_INFO: `/v3/users/9315aedb45514abfa86265acfc5eae30` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:07.279640 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63516dd-4e87-4c1b-93d7-bcd474764d83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9315aedb45514abfa86265acfc5eae30)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:07.281528 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: WARNING py.warnings [None req-e63516dd-4e87-4c1b-93d7-bcd474764d83 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 06:26:07.281528 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: warnings.warn(msg) Jul 18 06:26:07.281528 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: Jul 18 06:26:07.284454 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63516dd-4e87-4c1b-93d7-bcd474764d83 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:07.290413 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 833/1642] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:06 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:07.349137 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61e3f0-4985-450c-9aee-dec612228ed9 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:07.465352 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:07.583902 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61e3f0-4985-450c-9aee-dec612228ed9 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:07.601717 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ea5e34-f01e-45b0-9b97-7fa2a70d67c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:07.602984 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea5e34-f01e-45b0-9b97-7fa2a70d67c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:07.603807 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea5e34-f01e-45b0-9b97-7fa2a70d67c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:07.604278 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea5e34-f01e-45b0-9b97-7fa2a70d67c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:07.665185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ea5e34-f01e-45b0-9b97-7fa2a70d67c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:07.676400 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ea5e34-f01e-45b0-9b97-7fa2a70d67c4 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:07.743304 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 834/1643] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:07 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:07.753788 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e63516dd-4e87-4c1b-93d7-bcd474764d83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9315aedb45514abfa86265acfc5eae30'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:07.774933 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e63516dd-4e87-4c1b-93d7-bcd474764d83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9315aedb45514abfa86265acfc5eae30'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:07.806457 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e63516dd-4e87-4c1b-93d7-bcd474764d83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9315aedb45514abfa86265acfc5eae30'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:07.807604 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e63516dd-4e87-4c1b-93d7-bcd474764d83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9315aedb45514abfa86265acfc5eae30'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:07.808551 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e63516dd-4e87-4c1b-93d7-bcd474764d83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9315aedb45514abfa86265acfc5eae30'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:07.876787 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ea5e34-f01e-45b0-9b97-7fa2a70d67c4 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:07.905574 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ea5e34-f01e-45b0-9b97-7fa2a70d67c4 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:07.918144 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d9db1-a8cc-427d-af23-d8afb60bf3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:07.919712 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d9db1-a8cc-427d-af23-d8afb60bf3f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:07.920196 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d9db1-a8cc-427d-af23-d8afb60bf3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:07.920749 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d9db1-a8cc-427d-af23-d8afb60bf3f9 admin admin] PATH_INFO: `/v3/users/11e2d1981b984919903cbbba65963c91` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:07.923107 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75d9db1-a8cc-427d-af23-d8afb60bf3f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11e2d1981b984919903cbbba65963c91)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:07.945412 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75d9db1-a8cc-427d-af23-d8afb60bf3f9 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:07.980760 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 810/1644] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:26:07 2022] DELETE /identity/v3/users/9315aedb45514abfa86265acfc5eae30 => generated 0 bytes in 930 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:07.990607 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f5cb4-9c7f-4d62-a610-88a4faf156ae None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:07.991389 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f5cb4-9c7f-4d62-a610-88a4faf156ae None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:07.991873 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f5cb4-9c7f-4d62-a610-88a4faf156ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:08.082773 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-3f8f5cb4-9c7f-4d62-a610-88a4faf156ae None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:08.084648 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-3f8f5cb4-9c7f-4d62-a610-88a4faf156ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:08.281696 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:08.293389 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 811/1645] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:08.294330 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f5cb4-9c7f-4d62-a610-88a4faf156ae None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:08.313151 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-3f8f5cb4-9c7f-4d62-a610-88a4faf156ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:08.319552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-f75d9db1-a8cc-427d-af23-d8afb60bf3f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11e2d1981b984919903cbbba65963c91'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:08.339902 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-f75d9db1-a8cc-427d-af23-d8afb60bf3f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11e2d1981b984919903cbbba65963c91'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:08.344369 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-f75d9db1-a8cc-427d-af23-d8afb60bf3f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11e2d1981b984919903cbbba65963c91'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:08.346998 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-f75d9db1-a8cc-427d-af23-d8afb60bf3f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11e2d1981b984919903cbbba65963c91'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:08.354220 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f5cb4-9c7f-4d62-a610-88a4faf156ae None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:08.368482 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e5a414-237a-44e7-9e32-63072aa84013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:08.371426 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5a414-237a-44e7-9e32-63072aa84013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:08.371426 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5a414-237a-44e7-9e32-63072aa84013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:08.371426 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5a414-237a-44e7-9e32-63072aa84013 admin admin] PATH_INFO: `/v3/users/79ac4b77d2704391b0512d9c80995342` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:08.373347 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e5a414-237a-44e7-9e32-63072aa84013 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79ac4b77d2704391b0512d9c80995342)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:08.381283 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-f75d9db1-a8cc-427d-af23-d8afb60bf3f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11e2d1981b984919903cbbba65963c91'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:08.382472 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e5a414-237a-44e7-9e32-63072aa84013 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:08.553540 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 835/1646] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:26:07 2022] DELETE /identity/v3/users/11e2d1981b984919903cbbba65963c91 => generated 0 bytes in 681 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:08.559622 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d9db1-a8cc-427d-af23-d8afb60bf3f9 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:08.563350 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-f75d9db1-a8cc-427d-af23-d8afb60bf3f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:08.713621 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-94e5a414-237a-44e7-9e32-63072aa84013 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79ac4b77d2704391b0512d9c80995342'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:08.738468 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-94e5a414-237a-44e7-9e32-63072aa84013 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79ac4b77d2704391b0512d9c80995342'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:08.744504 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d9db1-a8cc-427d-af23-d8afb60bf3f9 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:08.766388 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f1b7c0-727f-4366-b894-ccb2bb0dacbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:08.767583 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1b7c0-727f-4366-b894-ccb2bb0dacbf service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:08.768092 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1b7c0-727f-4366-b894-ccb2bb0dacbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:08.768622 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1b7c0-727f-4366-b894-ccb2bb0dacbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:08.781295 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-a7f1b7c0-727f-4366-b894-ccb2bb0dacbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:08.787836 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-94e5a414-237a-44e7-9e32-63072aa84013 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79ac4b77d2704391b0512d9c80995342'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:08.789324 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-94e5a414-237a-44e7-9e32-63072aa84013 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79ac4b77d2704391b0512d9c80995342'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:08.791691 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-94e5a414-237a-44e7-9e32-63072aa84013 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79ac4b77d2704391b0512d9c80995342'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:08.912368 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:08.921570 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 812/1647] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:26:08 2022] DELETE /identity/v3/users/79ac4b77d2704391b0512d9c80995342 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:08.923242 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488a99a-bb3a-46e6-bacc-df55e9e719ab None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:08.923546 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488a99a-bb3a-46e6-bacc-df55e9e719ab None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:08.923830 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488a99a-bb3a-46e6-bacc-df55e9e719ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:08.992486 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-d488a99a-bb3a-46e6-bacc-df55e9e719ab None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:08.998194 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-d488a99a-bb3a-46e6-bacc-df55e9e719ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:09.028200 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f1b7c0-727f-4366-b894-ccb2bb0dacbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:09.034554 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f1b7c0-727f-4366-b894-ccb2bb0dacbf service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:09.067757 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 836/1648] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:09.070179 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310c232-4345-4ff4-8ad5-f0691946a309 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:09.070939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310c232-4345-4ff4-8ad5-f0691946a309 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:09.071432 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310c232-4345-4ff4-8ad5-f0691946a309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:09.150807 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-a310c232-4345-4ff4-8ad5-f0691946a309 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:09.163378 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 813/1649] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:09.168176 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-a310c232-4345-4ff4-8ad5-f0691946a309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:09.180836 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d488a99a-bb3a-46e6-bacc-df55e9e719ab None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:09.185429 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-d488a99a-bb3a-46e6-bacc-df55e9e719ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:09.214420 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 837/1650] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:09.272551 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310c232-4345-4ff4-8ad5-f0691946a309 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:09.276175 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-a310c232-4345-4ff4-8ad5-f0691946a309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:09.321372 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d488a99a-bb3a-46e6-bacc-df55e9e719ab None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:09.329512 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c146d2-fa6f-4c80-ab2d-bde6b7fd8b3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:09.330710 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c146d2-fa6f-4c80-ab2d-bde6b7fd8b3d service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:09.331909 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c146d2-fa6f-4c80-ab2d-bde6b7fd8b3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:09.332434 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c146d2-fa6f-4c80-ab2d-bde6b7fd8b3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:09.361432 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c146d2-fa6f-4c80-ab2d-bde6b7fd8b3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:09.367608 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c146d2-fa6f-4c80-ab2d-bde6b7fd8b3d service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:09.416910 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 814/1651] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 06:26:09.425865 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310c232-4345-4ff4-8ad5-f0691946a309 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:09.436600 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c174764-e664-4d02-9357-49db1887ec22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:09.437666 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c174764-e664-4d02-9357-49db1887ec22 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:09.439165 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c174764-e664-4d02-9357-49db1887ec22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:09.439724 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c174764-e664-4d02-9357-49db1887ec22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:09.475130 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c174764-e664-4d02-9357-49db1887ec22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:09.493624 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c174764-e664-4d02-9357-49db1887ec22 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:09.531117 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:09.543583 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 838/1652] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:10.867256 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c146d2-fa6f-4c80-ab2d-bde6b7fd8b3d service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:10.881649 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-95c146d2-fa6f-4c80-ab2d-bde6b7fd8b3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:10.921183 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c146d2-fa6f-4c80-ab2d-bde6b7fd8b3d service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:10.932390 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d4115-0cc8-4cf2-a0c9-360b8de23406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:10.936224 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d4115-0cc8-4cf2-a0c9-360b8de23406 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:10.937213 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d4115-0cc8-4cf2-a0c9-360b8de23406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:10.937963 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d4115-0cc8-4cf2-a0c9-360b8de23406 admin admin] PATH_INFO: `/v3/projects/404c5f366b1c49bab19fbf26cadf7029` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:10.941122 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4d4115-0cc8-4cf2-a0c9-360b8de23406 admin admin] RBAC: Authorizing `identity:delete_project(project_id=404c5f366b1c49bab19fbf26cadf7029)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:10.946325 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4d4115-0cc8-4cf2-a0c9-360b8de23406 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:11.133944 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-de4d4115-0cc8-4cf2-a0c9-360b8de23406 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '404c5f366b1c49bab19fbf26cadf7029', 'request_id': 'req-de4d4115-0cc8-4cf2-a0c9-360b8de23406'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:11.148957 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-de4d4115-0cc8-4cf2-a0c9-360b8de23406 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '404c5f366b1c49bab19fbf26cadf7029', 'request_id': 'req-de4d4115-0cc8-4cf2-a0c9-360b8de23406'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:11.332512 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-de4d4115-0cc8-4cf2-a0c9-360b8de23406 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '404c5f366b1c49bab19fbf26cadf7029', 'request_id': 'req-de4d4115-0cc8-4cf2-a0c9-360b8de23406'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:11.351029 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-de4d4115-0cc8-4cf2-a0c9-360b8de23406 admin admin] The token cache is being invalidate because project 404c5f366b1c49bab19fbf26cadf7029 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:26:11.351440 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-de4d4115-0cc8-4cf2-a0c9-360b8de23406 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:11.352662 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-de4d4115-0cc8-4cf2-a0c9-360b8de23406 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:11.355288 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 815/1653] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:26:10 2022] DELETE /identity/v3/projects/404c5f366b1c49bab19fbf26cadf7029 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:11.361289 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c174764-e664-4d02-9357-49db1887ec22 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:11.388054 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c174764-e664-4d02-9357-49db1887ec22 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:11.446737 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d4115-0cc8-4cf2-a0c9-360b8de23406 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:11.449463 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-de4d4115-0cc8-4cf2-a0c9-360b8de23406 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:11.571858 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccaecc9-12a1-436a-90e0-67154086a144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:11.577989 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccaecc9-12a1-436a-90e0-67154086a144 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:11.579330 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccaecc9-12a1-436a-90e0-67154086a144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:11.579814 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccaecc9-12a1-436a-90e0-67154086a144 admin admin] PATH_INFO: `/v3/users/8c58dfcd728b4636a539c630f864bccd` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:11.581483 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:11.587043 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccaecc9-12a1-436a-90e0-67154086a144 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c58dfcd728b4636a539c630f864bccd)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:11.596179 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccaecc9-12a1-436a-90e0-67154086a144 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:11.783478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d4115-0cc8-4cf2-a0c9-360b8de23406 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:11.793996 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a15fd6-6386-471f-b6c4-75c16bfef71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:11.795720 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a15fd6-6386-471f-b6c4-75c16bfef71f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:11.796381 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a15fd6-6386-471f-b6c4-75c16bfef71f admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:11.796734 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a15fd6-6386-471f-b6c4-75c16bfef71f admin admin] PATH_INFO: `/v3/projects/da1cc5ce226a40a7ab6a90e08f43a0e4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:11.806845 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a15fd6-6386-471f-b6c4-75c16bfef71f admin admin] RBAC: Authorizing `identity:delete_project(project_id=da1cc5ce226a40a7ab6a90e08f43a0e4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:11.817437 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a15fd6-6386-471f-b6c4-75c16bfef71f admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:11.967763 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8ccaecc9-12a1-436a-90e0-67154086a144 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c58dfcd728b4636a539c630f864bccd'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:11.992581 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8ccaecc9-12a1-436a-90e0-67154086a144 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c58dfcd728b4636a539c630f864bccd'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:11.994203 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8ccaecc9-12a1-436a-90e0-67154086a144 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c58dfcd728b4636a539c630f864bccd'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:12.019413 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8ccaecc9-12a1-436a-90e0-67154086a144 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c58dfcd728b4636a539c630f864bccd'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:12.111365 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-80a15fd6-6386-471f-b6c4-75c16bfef71f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da1cc5ce226a40a7ab6a90e08f43a0e4', 'request_id': 'req-80a15fd6-6386-471f-b6c4-75c16bfef71f'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:12.155380 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-80a15fd6-6386-471f-b6c4-75c16bfef71f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da1cc5ce226a40a7ab6a90e08f43a0e4', 'request_id': 'req-80a15fd6-6386-471f-b6c4-75c16bfef71f'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:12.173766 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8ccaecc9-12a1-436a-90e0-67154086a144 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c58dfcd728b4636a539c630f864bccd'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:12.242986 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-80a15fd6-6386-471f-b6c4-75c16bfef71f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da1cc5ce226a40a7ab6a90e08f43a0e4', 'request_id': 'req-80a15fd6-6386-471f-b6c4-75c16bfef71f'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:12.320206 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-80a15fd6-6386-471f-b6c4-75c16bfef71f admin admin] The token cache is being invalidate because project da1cc5ce226a40a7ab6a90e08f43a0e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:26:12.323837 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-80a15fd6-6386-471f-b6c4-75c16bfef71f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:12.329754 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-80a15fd6-6386-471f-b6c4-75c16bfef71f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:12.348172 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 816/1654] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:26:11 2022] DELETE /identity/v3/projects/da1cc5ce226a40a7ab6a90e08f43a0e4 => generated 0 bytes in 902 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:26:12.349955 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a15fd6-6386-471f-b6c4-75c16bfef71f admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:12.359505 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-80a15fd6-6386-471f-b6c4-75c16bfef71f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:12.359820 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 839/1655] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:26:11 2022] DELETE /identity/v3/users/8c58dfcd728b4636a539c630f864bccd => generated 0 bytes in 997 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:12.361442 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63340d71-23b9-4d8d-a4d4-783d657ca3ca None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:12.362400 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63340d71-23b9-4d8d-a4d4-783d657ca3ca None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:12.362400 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63340d71-23b9-4d8d-a4d4-783d657ca3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:12.409075 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-63340d71-23b9-4d8d-a4d4-783d657ca3ca None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:12.410385 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-63340d71-23b9-4d8d-a4d4-783d657ca3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:12.600632 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 840/1656] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:12.601680 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63340d71-23b9-4d8d-a4d4-783d657ca3ca None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:12.607752 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-63340d71-23b9-4d8d-a4d4-783d657ca3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:12.636598 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63340d71-23b9-4d8d-a4d4-783d657ca3ca None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:12.649976 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980fe7e3-c23d-4f1c-adda-af2fa32cb9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:12.650746 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fe7e3-c23d-4f1c-adda-af2fa32cb9ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:12.655505 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fe7e3-c23d-4f1c-adda-af2fa32cb9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:12.655934 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fe7e3-c23d-4f1c-adda-af2fa32cb9ca admin admin] PATH_INFO: `/v3/users/693a660dbdb4442e874373978651f295` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:12.660555 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980fe7e3-c23d-4f1c-adda-af2fa32cb9ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=693a660dbdb4442e874373978651f295)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:12.671648 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980fe7e3-c23d-4f1c-adda-af2fa32cb9ca admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:12.717136 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a15fd6-6386-471f-b6c4-75c16bfef71f admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:12.738154 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7e919-e4d9-4aa1-9adf-eae9ef00b7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:12.739910 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7e919-e4d9-4aa1-9adf-eae9ef00b7af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:12.740230 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7e919-e4d9-4aa1-9adf-eae9ef00b7af admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:12.742185 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7e919-e4d9-4aa1-9adf-eae9ef00b7af admin admin] PATH_INFO: `/v3/projects/0f5c0c9873314a8baff2d32bf64c162a` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:12.745671 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:12.747708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7e919-e4d9-4aa1-9adf-eae9ef00b7af admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f5c0c9873314a8baff2d32bf64c162a)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:12.760298 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7e919-e4d9-4aa1-9adf-eae9ef00b7af admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:13.382721 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9ba7e919-e4d9-4aa1-9adf-eae9ef00b7af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f5c0c9873314a8baff2d32bf64c162a', 'request_id': 'req-9ba7e919-e4d9-4aa1-9adf-eae9ef00b7af'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:13.393962 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:13.401568 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-980fe7e3-c23d-4f1c-adda-af2fa32cb9ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '693a660dbdb4442e874373978651f295'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:13.409346 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9ba7e919-e4d9-4aa1-9adf-eae9ef00b7af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f5c0c9873314a8baff2d32bf64c162a', 'request_id': 'req-9ba7e919-e4d9-4aa1-9adf-eae9ef00b7af'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:13.435220 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-980fe7e3-c23d-4f1c-adda-af2fa32cb9ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '693a660dbdb4442e874373978651f295'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:13.453090 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-980fe7e3-c23d-4f1c-adda-af2fa32cb9ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '693a660dbdb4442e874373978651f295'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:13.453656 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-980fe7e3-c23d-4f1c-adda-af2fa32cb9ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '693a660dbdb4442e874373978651f295'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:13.482435 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-980fe7e3-c23d-4f1c-adda-af2fa32cb9ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '693a660dbdb4442e874373978651f295'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:13.539046 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9ba7e919-e4d9-4aa1-9adf-eae9ef00b7af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f5c0c9873314a8baff2d32bf64c162a', 'request_id': 'req-9ba7e919-e4d9-4aa1-9adf-eae9ef00b7af'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:13.559440 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9ba7e919-e4d9-4aa1-9adf-eae9ef00b7af admin admin] The token cache is being invalidate because project 0f5c0c9873314a8baff2d32bf64c162a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:26:13.559887 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9ba7e919-e4d9-4aa1-9adf-eae9ef00b7af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:13.561348 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9ba7e919-e4d9-4aa1-9adf-eae9ef00b7af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:13.568895 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 817/1657] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:26:12 2022] DELETE /identity/v3/projects/0f5c0c9873314a8baff2d32bf64c162a => generated 0 bytes in 1216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:13.572538 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d7c59-ecb8-46fc-8a05-bc0f9627e4ab None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:13.573000 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d7c59-ecb8-46fc-8a05-bc0f9627e4ab None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:13.573772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d7c59-ecb8-46fc-8a05-bc0f9627e4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:13.598713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-469d7c59-ecb8-46fc-8a05-bc0f9627e4ab None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:13.600705 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-469d7c59-ecb8-46fc-8a05-bc0f9627e4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:13.613518 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 841/1658] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:26:12 2022] DELETE /identity/v3/users/693a660dbdb4442e874373978651f295 => generated 0 bytes in 1012 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:13.617882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980fe7e3-c23d-4f1c-adda-af2fa32cb9ca admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:13.619122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-980fe7e3-c23d-4f1c-adda-af2fa32cb9ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:13.731941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 818/1659] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:13.731941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c9483-d89d-4e2e-b93d-e0d85ed86bb8 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:13.731941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c9483-d89d-4e2e-b93d-e0d85ed86bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:13.731941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c9483-d89d-4e2e-b93d-e0d85ed86bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:13.757601 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-ef4c9483-d89d-4e2e-b93d-e0d85ed86bb8 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:13.760222 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ef4c9483-d89d-4e2e-b93d-e0d85ed86bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:13.856054 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 819/1660] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:13.861424 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c9483-d89d-4e2e-b93d-e0d85ed86bb8 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:13.866032 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980fe7e3-c23d-4f1c-adda-af2fa32cb9ca admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:13.872286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c9483-d89d-4e2e-b93d-e0d85ed86bb8 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:13.880964 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e2574-2f5c-47d1-a584-d26d471b33bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:13.886721 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e2574-2f5c-47d1-a584-d26d471b33bd admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:13.892299 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e2574-2f5c-47d1-a584-d26d471b33bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:13.892299 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e2574-2f5c-47d1-a584-d26d471b33bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:13.892299 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e2574-2f5c-47d1-a584-d26d471b33bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:26:13.892299 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e2574-2f5c-47d1-a584-d26d471b33bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:13.895454 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e2574-2f5c-47d1-a584-d26d471b33bd admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:13.897037 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda3f34-7d2f-47e8-a303-48f4c57b732e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:13.899749 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda3f34-7d2f-47e8-a303-48f4c57b732e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:13.900954 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda3f34-7d2f-47e8-a303-48f4c57b732e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:13.900954 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda3f34-7d2f-47e8-a303-48f4c57b732e admin admin] PATH_INFO: `/v3/users/f0380a338e824375bab94af7cded3ea6` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:13.905198 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda3f34-7d2f-47e8-a303-48f4c57b732e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0380a338e824375bab94af7cded3ea6)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:13.914516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda3f34-7d2f-47e8-a303-48f4c57b732e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:13.944853 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 820/1661] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:26:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:26:13.944853 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e2574-2f5c-47d1-a584-d26d471b33bd admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:13.948254 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-db6e2574-2f5c-47d1-a584-d26d471b33bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:14.139743 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e2574-2f5c-47d1-a584-d26d471b33bd admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:14.151917 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cf118c-139f-4f1b-9c18-4e1d537444aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:14.156164 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf118c-139f-4f1b-9c18-4e1d537444aa service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:14.157851 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf118c-139f-4f1b-9c18-4e1d537444aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:14.158586 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf118c-139f-4f1b-9c18-4e1d537444aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:14.192144 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cf118c-139f-4f1b-9c18-4e1d537444aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:14.209610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cf118c-139f-4f1b-9c18-4e1d537444aa service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:14.247837 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 821/1662] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:14.249697 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cf118c-139f-4f1b-9c18-4e1d537444aa service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:14.250063 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4fda3f34-7d2f-47e8-a303-48f4c57b732e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0380a338e824375bab94af7cded3ea6'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:14.281515 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4fda3f34-7d2f-47e8-a303-48f4c57b732e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0380a338e824375bab94af7cded3ea6'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:14.281807 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cf118c-139f-4f1b-9c18-4e1d537444aa service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:14.284286 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4fda3f34-7d2f-47e8-a303-48f4c57b732e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0380a338e824375bab94af7cded3ea6'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:14.289298 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4fda3f34-7d2f-47e8-a303-48f4c57b732e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0380a338e824375bab94af7cded3ea6'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:14.293483 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc012a78-0814-423d-bbf8-586e77fb2024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:14.301231 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc012a78-0814-423d-bbf8-586e77fb2024 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:14.301231 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc012a78-0814-423d-bbf8-586e77fb2024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:14.301231 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc012a78-0814-423d-bbf8-586e77fb2024 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:14.301231 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc012a78-0814-423d-bbf8-586e77fb2024 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:14.303049 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc012a78-0814-423d-bbf8-586e77fb2024 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:14.340770 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4fda3f34-7d2f-47e8-a303-48f4c57b732e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0380a338e824375bab94af7cded3ea6'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:14.461332 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 842/1663] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:26:13 2022] DELETE /identity/v3/users/f0380a338e824375bab94af7cded3ea6 => generated 0 bytes in 847 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:14.467781 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda3f34-7d2f-47e8-a303-48f4c57b732e admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:14.469734 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4fda3f34-7d2f-47e8-a303-48f4c57b732e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:14.471624 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 822/1664] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:26:14 2022] POST /identity/v3/projects => generated 363 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:14.484792 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc012a78-0814-423d-bbf8-586e77fb2024 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:14.493275 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-cc012a78-0814-423d-bbf8-586e77fb2024 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:14.708466 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda3f34-7d2f-47e8-a303-48f4c57b732e admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:14.776809 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc012a78-0814-423d-bbf8-586e77fb2024 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:14.793617 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a59ce1-440d-431c-801d-58e7e65e281f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:14.794156 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a59ce1-440d-431c-801d-58e7e65e281f admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:14.794314 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a59ce1-440d-431c-801d-58e7e65e281f admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:14.794603 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a59ce1-440d-431c-801d-58e7e65e281f admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:14.795357 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a59ce1-440d-431c-801d-58e7e65e281f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:14.803057 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a59ce1-440d-431c-801d-58e7e65e281f admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:14.814180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011087ca-8fd9-4520-acbd-3d08e3e69b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:14.821953 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011087ca-8fd9-4520-acbd-3d08e3e69b54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:14.834976 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011087ca-8fd9-4520-acbd-3d08e3e69b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:14.835631 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011087ca-8fd9-4520-acbd-3d08e3e69b54 admin admin] PATH_INFO: `/v3/projects/32450e66e64c449288866dd4cb52f939` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:14.855540 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011087ca-8fd9-4520-acbd-3d08e3e69b54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32450e66e64c449288866dd4cb52f939)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:14.872133 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: WARNING py.warnings [None req-011087ca-8fd9-4520-acbd-3d08e3e69b54 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 06:26:14.872133 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: warnings.warn(msg) Jul 18 06:26:14.872133 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: Jul 18 06:26:14.880764 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011087ca-8fd9-4520-acbd-3d08e3e69b54 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:14.973680 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 823/1665] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:26:14 2022] POST /identity/v3/users => generated 282 bytes in 490 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:26:14.981110 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a59ce1-440d-431c-801d-58e7e65e281f admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:15.018581 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a59ce1-440d-431c-801d-58e7e65e281f admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:15.032387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b5ef3f-64fd-42c0-82aa-400de675d41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:15.035931 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5ef3f-64fd-42c0-82aa-400de675d41f admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:15.036599 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5ef3f-64fd-42c0-82aa-400de675d41f admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:15.037261 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5ef3f-64fd-42c0-82aa-400de675d41f admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:15.041426 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b5ef3f-64fd-42c0-82aa-400de675d41f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:15.047271 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b5ef3f-64fd-42c0-82aa-400de675d41f admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:15.082883 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:15.093745 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 824/1666] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:15.103544 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b5ef3f-64fd-42c0-82aa-400de675d41f admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:15.172521 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b5ef3f-64fd-42c0-82aa-400de675d41f admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:15.197756 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-011087ca-8fd9-4520-acbd-3d08e3e69b54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32450e66e64c449288866dd4cb52f939', 'request_id': 'req-011087ca-8fd9-4520-acbd-3d08e3e69b54'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:15.257169 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-011087ca-8fd9-4520-acbd-3d08e3e69b54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32450e66e64c449288866dd4cb52f939', 'request_id': 'req-011087ca-8fd9-4520-acbd-3d08e3e69b54'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:15.369932 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-011087ca-8fd9-4520-acbd-3d08e3e69b54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32450e66e64c449288866dd4cb52f939', 'request_id': 'req-011087ca-8fd9-4520-acbd-3d08e3e69b54'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:15.401554 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-011087ca-8fd9-4520-acbd-3d08e3e69b54 admin admin] The token cache is being invalidate because project 32450e66e64c449288866dd4cb52f939 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:26:15.401926 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-011087ca-8fd9-4520-acbd-3d08e3e69b54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:15.405771 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-011087ca-8fd9-4520-acbd-3d08e3e69b54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:15.412722 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 843/1667] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:26:14 2022] DELETE /identity/v3/projects/32450e66e64c449288866dd4cb52f939 => generated 0 bytes in 949 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:15.420408 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c2663-9a8e-4513-bf60-bfe20c91f46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:15.421407 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c2663-9a8e-4513-bf60-bfe20c91f46f admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:15.421844 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c2663-9a8e-4513-bf60-bfe20c91f46f admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:15.422278 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c2663-9a8e-4513-bf60-bfe20c91f46f admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:15.423355 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c2663-9a8e-4513-bf60-bfe20c91f46f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:15.423898 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38072b2c-607f-4ca9-8790-419977bf0de7 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:15.425938 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38072b2c-607f-4ca9-8790-419977bf0de7 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:15.426423 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38072b2c-607f-4ca9-8790-419977bf0de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:15.430736 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c2663-9a8e-4513-bf60-bfe20c91f46f admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:15.447367 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 825/1668] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:15 2022] GET /identity/v3/roles => generated 1420 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:15.448849 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c2663-9a8e-4513-bf60-bfe20c91f46f admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:15.451117 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-421c2663-9a8e-4513-bf60-bfe20c91f46f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:15.459129 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-38072b2c-607f-4ca9-8790-419977bf0de7 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:15.462257 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-38072b2c-607f-4ca9-8790-419977bf0de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:15.477634 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 844/1669] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:15.482384 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c2663-9a8e-4513-bf60-bfe20c91f46f admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:15.486421 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e30274-caf6-43e2-95bd-2a5605a8f512 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:15.486888 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e30274-caf6-43e2-95bd-2a5605a8f512 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:15.487233 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e30274-caf6-43e2-95bd-2a5605a8f512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:15.500278 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef258d-ab30-4471-905c-c272f0d7c83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:15.500278 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef258d-ab30-4471-905c-c272f0d7c83c admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:15.500278 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef258d-ab30-4471-905c-c272f0d7c83c admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:15.500278 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef258d-ab30-4471-905c-c272f0d7c83c admin admin] PATH_INFO: `/v3/projects/41c528319bab4e1ea147f98b548142ff/users/d32b8a126e744671a81a63a9cb73ae21/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:15.520088 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-13e30274-caf6-43e2-95bd-2a5605a8f512 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:15.523300 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-13e30274-caf6-43e2-95bd-2a5605a8f512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:15.528547 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ef258d-ab30-4471-905c-c272f0d7c83c admin admin] RBAC: Authorizing `identity:create_grant(project_id=41c528319bab4e1ea147f98b548142ff, user_id=d32b8a126e744671a81a63a9cb73ae21, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:15.534259 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ef258d-ab30-4471-905c-c272f0d7c83c admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:15.536148 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 845/1670] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:26:15.540672 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e30274-caf6-43e2-95bd-2a5605a8f512 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:15.574409 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e30274-caf6-43e2-95bd-2a5605a8f512 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:15.584518 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f28754-d0b6-431f-a296-ea7223129329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:15.585135 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f28754-d0b6-431f-a296-ea7223129329 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:15.585438 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f28754-d0b6-431f-a296-ea7223129329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:15.585727 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f28754-d0b6-431f-a296-ea7223129329 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:15.586498 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f28754-d0b6-431f-a296-ea7223129329 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:26:15.586821 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f28754-d0b6-431f-a296-ea7223129329 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:15.588578 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f28754-d0b6-431f-a296-ea7223129329 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:15.615623 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 846/1671] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:26:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:26:15.617155 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f28754-d0b6-431f-a296-ea7223129329 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:15.635321 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f28754-d0b6-431f-a296-ea7223129329 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:15.643581 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98a96cb-bc74-41fe-bc5d-637e6870fbd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:15.644932 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a96cb-bc74-41fe-bc5d-637e6870fbd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:15.645479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a96cb-bc74-41fe-bc5d-637e6870fbd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:15.645948 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a96cb-bc74-41fe-bc5d-637e6870fbd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:15.647895 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98a96cb-bc74-41fe-bc5d-637e6870fbd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:15.663704 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98a96cb-bc74-41fe-bc5d-637e6870fbd8 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:15.670303 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 826/1672] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:15 2022] PUT /identity/v3/projects/41c528319bab4e1ea147f98b548142ff/users/d32b8a126e744671a81a63a9cb73ae21/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:15.683661 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659080e6-f4a3-49d4-a58c-4c5b1639c672 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:15.684692 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659080e6-f4a3-49d4-a58c-4c5b1639c672 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:15.685373 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659080e6-f4a3-49d4-a58c-4c5b1639c672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:15.769324 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-659080e6-f4a3-49d4-a58c-4c5b1639c672 None None] MFA Rules not processed for user `d32b8a126e744671a81a63a9cb73ae21`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:15.772435 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-659080e6-f4a3-49d4-a58c-4c5b1639c672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:15.852224 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 847/1673] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:26:15 2022] POST /identity/v3/projects => generated 381 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:15.853589 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98a96cb-bc74-41fe-bc5d-637e6870fbd8 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:15.962428 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:16.048153 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98a96cb-bc74-41fe-bc5d-637e6870fbd8 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:16.079688 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 827/1674] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:15 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:16.085309 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659080e6-f4a3-49d4-a58c-4c5b1639c672 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:16.111303 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8c77f-d88a-4127-afb0-15d8348c7ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:16.118121 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8c77f-d88a-4127-afb0-15d8348c7ff5 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:16.119086 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8c77f-d88a-4127-afb0-15d8348c7ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:16.124937 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8c77f-d88a-4127-afb0-15d8348c7ff5 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:16.130645 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8c77f-d88a-4127-afb0-15d8348c7ff5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:16.137090 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8c77f-d88a-4127-afb0-15d8348c7ff5 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:16.290024 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 848/1675] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:26:15 2022] POST /identity/v3/users => generated 291 bytes in 437 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:16.294400 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8c77f-d88a-4127-afb0-15d8348c7ff5 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:16.339635 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8c77f-d88a-4127-afb0-15d8348c7ff5 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:16.370989 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa3eee3-5093-4c48-8226-72fc8bb1be5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:16.378985 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa3eee3-5093-4c48-8226-72fc8bb1be5c admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:16.379593 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa3eee3-5093-4c48-8226-72fc8bb1be5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:16.380661 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa3eee3-5093-4c48-8226-72fc8bb1be5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:16.388125 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659080e6-f4a3-49d4-a58c-4c5b1639c672 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:16.389048 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa3eee3-5093-4c48-8226-72fc8bb1be5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:16.391637 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa3eee3-5093-4c48-8226-72fc8bb1be5c admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:16.444867 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b3030-a1a1-4fe1-8e29-3e7a82d7e3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:16.448030 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b3030-a1a1-4fe1-8e29-3e7a82d7e3cf admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:16.448626 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b3030-a1a1-4fe1-8e29-3e7a82d7e3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:16.456681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b3030-a1a1-4fe1-8e29-3e7a82d7e3cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:16.463209 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b3030-a1a1-4fe1-8e29-3e7a82d7e3cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:16.483111 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b3030-a1a1-4fe1-8e29-3e7a82d7e3cf admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:16.490092 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 849/1676] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:16 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:16.491859 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa3eee3-5093-4c48-8226-72fc8bb1be5c admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:16.515505 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa3eee3-5093-4c48-8226-72fc8bb1be5c admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:16.528076 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e311a0-7ab3-4a36-9c40-5b9ab8402862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:16.529147 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e311a0-7ab3-4a36-9c40-5b9ab8402862 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:16.529586 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e311a0-7ab3-4a36-9c40-5b9ab8402862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:16.529997 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e311a0-7ab3-4a36-9c40-5b9ab8402862 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:16.531580 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e311a0-7ab3-4a36-9c40-5b9ab8402862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:16.534249 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e311a0-7ab3-4a36-9c40-5b9ab8402862 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:16.563939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 850/1677] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:16 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 06:26:16.566392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e311a0-7ab3-4a36-9c40-5b9ab8402862 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:16.582983 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e311a0-7ab3-4a36-9c40-5b9ab8402862 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:16.586365 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc96e12e-e4fd-4b87-850c-147348892cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:16.587355 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96e12e-e4fd-4b87-850c-147348892cd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:16.587641 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96e12e-e4fd-4b87-850c-147348892cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:16.587977 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96e12e-e4fd-4b87-850c-147348892cd5 admin admin] PATH_INFO: `/v3/projects/19737acd3c0542bd9d0c8978b1d98657/users/53bda2c82b4f470eab9043035e3a44b0/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:16.643210 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 828/1678] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:26:16 2022] POST /identity/v3/projects => generated 363 bytes in 556 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:16.644610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b3030-a1a1-4fe1-8e29-3e7a82d7e3cf admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:16.654824 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc96e12e-e4fd-4b87-850c-147348892cd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19737acd3c0542bd9d0c8978b1d98657, user_id=53bda2c82b4f470eab9043035e3a44b0, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:16.668980 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc96e12e-e4fd-4b87-850c-147348892cd5 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:16.810481 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 851/1679] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:16 2022] PUT /identity/v3/projects/19737acd3c0542bd9d0c8978b1d98657/users/53bda2c82b4f470eab9043035e3a44b0/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:16.836689 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35364a2f-f40e-4497-a117-f94f01ae8756 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:16.837801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35364a2f-f40e-4497-a117-f94f01ae8756 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:16.839911 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35364a2f-f40e-4497-a117-f94f01ae8756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:17.043746 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-35364a2f-f40e-4497-a117-f94f01ae8756 None None] MFA Rules not processed for user `53bda2c82b4f470eab9043035e3a44b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:17.043746 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-35364a2f-f40e-4497-a117-f94f01ae8756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:17.092528 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b3030-a1a1-4fe1-8e29-3e7a82d7e3cf admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:17.390153 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:17.432007 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a402743-cf57-4098-b5f4-8f6a49266614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:17.445560 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a402743-cf57-4098-b5f4-8f6a49266614 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:17.446853 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a402743-cf57-4098-b5f4-8f6a49266614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:17.447304 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a402743-cf57-4098-b5f4-8f6a49266614 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:17.448295 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a402743-cf57-4098-b5f4-8f6a49266614 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:17.453311 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a402743-cf57-4098-b5f4-8f6a49266614 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:17.587409 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 829/1680] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:26:16 2022] POST /identity/v3/users => generated 288 bytes in 943 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:17.589650 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a402743-cf57-4098-b5f4-8f6a49266614 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:17.595093 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7a402743-cf57-4098-b5f4-8f6a49266614 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:17.644721 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a402743-cf57-4098-b5f4-8f6a49266614 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:17.667296 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 852/1681] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 828 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:17.667296 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35364a2f-f40e-4497-a117-f94f01ae8756 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:17.669154 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:17.672074 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14be0c-b57c-4518-bec4-11a1f89c9ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:17.673210 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14be0c-b57c-4518-bec4-11a1f89c9ab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:17.673523 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14be0c-b57c-4518-bec4-11a1f89c9ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:17.673887 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14be0c-b57c-4518-bec4-11a1f89c9ab4 admin admin] PATH_INFO: `/v3/projects/8b1da02b710246e48ee8f8a64b15b45f` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:17.675387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c14be0c-b57c-4518-bec4-11a1f89c9ab4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b1da02b710246e48ee8f8a64b15b45f)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:17.679001 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c14be0c-b57c-4518-bec4-11a1f89c9ab4 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:17.696381 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35364a2f-f40e-4497-a117-f94f01ae8756 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:17.722623 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f5a575-1129-4da5-b66b-25f4958c2c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:17.728579 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5a575-1129-4da5-b66b-25f4958c2c2c admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:17.734432 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5a575-1129-4da5-b66b-25f4958c2c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:17.734432 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5a575-1129-4da5-b66b-25f4958c2c2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:17.734432 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f5a575-1129-4da5-b66b-25f4958c2c2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:17.741575 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f5a575-1129-4da5-b66b-25f4958c2c2c admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:17.797830 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 853/1682] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:17 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:17.800301 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f5a575-1129-4da5-b66b-25f4958c2c2c admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:17.846846 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f5a575-1129-4da5-b66b-25f4958c2c2c admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:17.869577 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f73c24-e823-455c-83fd-920dd0a02f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:17.870313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f73c24-e823-455c-83fd-920dd0a02f54 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:17.870606 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f73c24-e823-455c-83fd-920dd0a02f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:17.870890 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f73c24-e823-455c-83fd-920dd0a02f54 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:17.871885 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f73c24-e823-455c-83fd-920dd0a02f54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:17.873948 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f73c24-e823-455c-83fd-920dd0a02f54 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:17.974498 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0c14be0c-b57c-4518-bec4-11a1f89c9ab4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b1da02b710246e48ee8f8a64b15b45f', 'request_id': 'req-0c14be0c-b57c-4518-bec4-11a1f89c9ab4'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:17.992951 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0c14be0c-b57c-4518-bec4-11a1f89c9ab4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b1da02b710246e48ee8f8a64b15b45f', 'request_id': 'req-0c14be0c-b57c-4518-bec4-11a1f89c9ab4'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:18.061903 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 854/1683] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:26:17 2022] POST /identity/v3/projects => generated 381 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:18.063259 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f73c24-e823-455c-83fd-920dd0a02f54 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:18.081767 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0c14be0c-b57c-4518-bec4-11a1f89c9ab4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b1da02b710246e48ee8f8a64b15b45f', 'request_id': 'req-0c14be0c-b57c-4518-bec4-11a1f89c9ab4'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:18.128915 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0c14be0c-b57c-4518-bec4-11a1f89c9ab4 admin admin] The token cache is being invalidate because project 8b1da02b710246e48ee8f8a64b15b45f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:26:18.131250 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0c14be0c-b57c-4518-bec4-11a1f89c9ab4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:18.136247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0c14be0c-b57c-4518-bec4-11a1f89c9ab4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:18.147358 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 830/1684] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:26:17 2022] DELETE /identity/v3/projects/8b1da02b710246e48ee8f8a64b15b45f => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:18.151639 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14be0c-b57c-4518-bec4-11a1f89c9ab4 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:18.157852 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-0c14be0c-b57c-4518-bec4-11a1f89c9ab4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:18.232494 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:18.303807 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f73c24-e823-455c-83fd-920dd0a02f54 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:18.326300 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa905ade-2221-4254-a297-62833a6f4c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:18.326300 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa905ade-2221-4254-a297-62833a6f4c72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:18.326300 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa905ade-2221-4254-a297-62833a6f4c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:18.326803 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa905ade-2221-4254-a297-62833a6f4c72 admin admin] PATH_INFO: `/v3/projects/6ec30bf602434097952a709fb5ca4665/users/cd9203211bb144adb5bf25effea87dc7/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:18.371200 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14be0c-b57c-4518-bec4-11a1f89c9ab4 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:18.402757 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a565b4-b60a-4385-86ef-4c4be6b2dbb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:18.413205 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a565b4-b60a-4385-86ef-4c4be6b2dbb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:18.413205 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a565b4-b60a-4385-86ef-4c4be6b2dbb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:18.414751 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a565b4-b60a-4385-86ef-4c4be6b2dbb3 admin admin] PATH_INFO: `/v3/projects/93582115dfa34cf494ca6fdd51b68357` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:18.421093 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a565b4-b60a-4385-86ef-4c4be6b2dbb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93582115dfa34cf494ca6fdd51b68357)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:18.423279 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa905ade-2221-4254-a297-62833a6f4c72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ec30bf602434097952a709fb5ca4665, user_id=cd9203211bb144adb5bf25effea87dc7, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:18.427709 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a565b4-b60a-4385-86ef-4c4be6b2dbb3 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:18.436489 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa905ade-2221-4254-a297-62833a6f4c72 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:18.504720 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:18.540324 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 855/1685] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:18 2022] PUT /identity/v3/projects/6ec30bf602434097952a709fb5ca4665/users/cd9203211bb144adb5bf25effea87dc7/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:18.541696 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa905ade-2221-4254-a297-62833a6f4c72 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:18.543809 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-fa905ade-2221-4254-a297-62833a6f4c72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:18.689933 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-c5a565b4-b60a-4385-86ef-4c4be6b2dbb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93582115dfa34cf494ca6fdd51b68357', 'request_id': 'req-c5a565b4-b60a-4385-86ef-4c4be6b2dbb3'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:18.725984 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-c5a565b4-b60a-4385-86ef-4c4be6b2dbb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93582115dfa34cf494ca6fdd51b68357', 'request_id': 'req-c5a565b4-b60a-4385-86ef-4c4be6b2dbb3'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:18.732271 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa905ade-2221-4254-a297-62833a6f4c72 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:18.839838 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-c5a565b4-b60a-4385-86ef-4c4be6b2dbb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93582115dfa34cf494ca6fdd51b68357', 'request_id': 'req-c5a565b4-b60a-4385-86ef-4c4be6b2dbb3'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:18.871293 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-c5a565b4-b60a-4385-86ef-4c4be6b2dbb3 admin admin] The token cache is being invalidate because project 93582115dfa34cf494ca6fdd51b68357 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:26:18.871898 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-c5a565b4-b60a-4385-86ef-4c4be6b2dbb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:18.877451 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-c5a565b4-b60a-4385-86ef-4c4be6b2dbb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:18.886055 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 831/1686] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:26:18 2022] DELETE /identity/v3/projects/93582115dfa34cf494ca6fdd51b68357 => generated 0 bytes in 740 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:18.886401 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28600d4f-eab9-492c-9b8e-14f0af861930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:18.887405 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28600d4f-eab9-492c-9b8e-14f0af861930 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:18.887961 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28600d4f-eab9-492c-9b8e-14f0af861930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:18.889586 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28600d4f-eab9-492c-9b8e-14f0af861930 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:18.889956 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037cac5-7fdc-47bf-b4e4-ca067ebd2e98 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:18.890252 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037cac5-7fdc-47bf-b4e4-ca067ebd2e98 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:18.891575 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28600d4f-eab9-492c-9b8e-14f0af861930 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:18.896166 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037cac5-7fdc-47bf-b4e4-ca067ebd2e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:18.902275 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28600d4f-eab9-492c-9b8e-14f0af861930 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:18.968031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-b037cac5-7fdc-47bf-b4e4-ca067ebd2e98 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:18.969325 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b037cac5-7fdc-47bf-b4e4-ca067ebd2e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:19.002917 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 832/1687] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:19.003940 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b037cac5-7fdc-47bf-b4e4-ca067ebd2e98 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:19.006305 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b037cac5-7fdc-47bf-b4e4-ca067ebd2e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:19.025852 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 856/1688] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:26:18 2022] POST /identity/v3/users => generated 297 bytes in 485 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:19.032301 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bccc2-8877-4c36-9795-7581e79f6e17 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:19.032619 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bccc2-8877-4c36-9795-7581e79f6e17 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:19.033895 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:19.034581 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bccc2-8877-4c36-9795-7581e79f6e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:19.052447 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b037cac5-7fdc-47bf-b4e4-ca067ebd2e98 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:19.085079 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a63598-d453-4e4b-a928-dc17604cd0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:19.085786 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a63598-d453-4e4b-a928-dc17604cd0be admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:19.086074 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a63598-d453-4e4b-a928-dc17604cd0be admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:19.086350 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a63598-d453-4e4b-a928-dc17604cd0be admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:19.089988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a63598-d453-4e4b-a928-dc17604cd0be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:19.109177 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a63598-d453-4e4b-a928-dc17604cd0be admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:19.132755 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 833/1689] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:19 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:19.134708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff70c26-5b26-4c5c-83b3-3cb8b6344e7b None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:19.135238 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff70c26-5b26-4c5c-83b3-3cb8b6344e7b None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:19.135737 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff70c26-5b26-4c5c-83b3-3cb8b6344e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:19.138940 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-425bccc2-8877-4c36-9795-7581e79f6e17 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:19.148859 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-425bccc2-8877-4c36-9795-7581e79f6e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:19.181550 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 857/1690] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:19.183001 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425bccc2-8877-4c36-9795-7581e79f6e17 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:19.183401 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-4ff70c26-5b26-4c5c-83b3-3cb8b6344e7b None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:19.185132 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4ff70c26-5b26-4c5c-83b3-3cb8b6344e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:19.186885 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-425bccc2-8877-4c36-9795-7581e79f6e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:19.206363 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 834/1691] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:19.210194 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff70c26-5b26-4c5c-83b3-3cb8b6344e7b None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:19.223137 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425bccc2-8877-4c36-9795-7581e79f6e17 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:19.265997 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff70c26-5b26-4c5c-83b3-3cb8b6344e7b None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:19.271566 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec99e6-7b2d-4bf0-be8c-ab00d39550a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:19.272218 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec99e6-7b2d-4bf0-be8c-ab00d39550a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:19.272891 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec99e6-7b2d-4bf0-be8c-ab00d39550a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:19.273377 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec99e6-7b2d-4bf0-be8c-ab00d39550a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:19.286837 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ec99e6-7b2d-4bf0-be8c-ab00d39550a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:19.289495 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ec99e6-7b2d-4bf0-be8c-ab00d39550a7 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:19.289889 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccb8f03-b53e-4863-96bd-1eda9454718d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:19.291063 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb8f03-b53e-4863-96bd-1eda9454718d admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:19.291667 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb8f03-b53e-4863-96bd-1eda9454718d admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:19.292105 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb8f03-b53e-4863-96bd-1eda9454718d admin admin] PATH_INFO: `/v3/projects/6ec30bf602434097952a709fb5ca4665/users/cd9203211bb144adb5bf25effea87dc7/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:19.302254 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccb8f03-b53e-4863-96bd-1eda9454718d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ec30bf602434097952a709fb5ca4665, user_id=cd9203211bb144adb5bf25effea87dc7, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:19.312216 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccb8f03-b53e-4863-96bd-1eda9454718d admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:19.367920 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 835/1692] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:19 2022] PUT /identity/v3/projects/6ec30bf602434097952a709fb5ca4665/users/cd9203211bb144adb5bf25effea87dc7/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:19.376201 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 858/1693] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:19 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:19.376201 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec99e6-7b2d-4bf0-be8c-ab00d39550a7 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:19.382349 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7933ff-b0af-45a4-be10-4581a0fc87a3 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:19.385313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:19.388392 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7933ff-b0af-45a4-be10-4581a0fc87a3 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:19.389055 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7933ff-b0af-45a4-be10-4581a0fc87a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:19.399072 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec99e6-7b2d-4bf0-be8c-ab00d39550a7 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:19.464972 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-ba7933ff-b0af-45a4-be10-4581a0fc87a3 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:19.466257 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ba7933ff-b0af-45a4-be10-4581a0fc87a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:19.638053 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c82e4f2-2ea1-4f4b-8efd-3c9f78da795d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:19.638053 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82e4f2-2ea1-4f4b-8efd-3c9f78da795d admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:19.638053 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82e4f2-2ea1-4f4b-8efd-3c9f78da795d admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:19.638053 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82e4f2-2ea1-4f4b-8efd-3c9f78da795d admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:19.638539 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c82e4f2-2ea1-4f4b-8efd-3c9f78da795d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:26:19.638539 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c82e4f2-2ea1-4f4b-8efd-3c9f78da795d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:19.640215 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c82e4f2-2ea1-4f4b-8efd-3c9f78da795d admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:19.704199 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 836/1694] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:19.704199 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7933ff-b0af-45a4-be10-4581a0fc87a3 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:19.720558 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 859/1695] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:26:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:26:19.720558 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b70ae-117e-4267-8987-7185a80bf334 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:19.720558 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b70ae-117e-4267-8987-7185a80bf334 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:19.720558 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b70ae-117e-4267-8987-7185a80bf334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:19.730980 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7933ff-b0af-45a4-be10-4581a0fc87a3 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:19.764259 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62a3a14-75ef-426d-a862-ecca4f5900ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:19.766523 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a3a14-75ef-426d-a862-ecca4f5900ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:19.766839 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a3a14-75ef-426d-a862-ecca4f5900ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:19.771208 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a3a14-75ef-426d-a862-ecca4f5900ce admin admin] PATH_INFO: `/v3/projects/adaf93a38977429db9cfae6ee619e0a8/users/a378fd60c6af4778a443a21f5bcca46a/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:19.843046 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-d24b70ae-117e-4267-8987-7185a80bf334 None None] MFA Rules not processed for user `cd9203211bb144adb5bf25effea87dc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:19.844641 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d24b70ae-117e-4267-8987-7185a80bf334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:19.901344 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62a3a14-75ef-426d-a862-ecca4f5900ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=adaf93a38977429db9cfae6ee619e0a8, user_id=a378fd60c6af4778a443a21f5bcca46a, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:19.920085 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62a3a14-75ef-426d-a862-ecca4f5900ce admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:20.053726 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 837/1696] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:19 2022] PUT /identity/v3/projects/adaf93a38977429db9cfae6ee619e0a8/users/a378fd60c6af4778a443a21f5bcca46a/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:20.055288 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62a3a14-75ef-426d-a862-ecca4f5900ce admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:20.069948 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 860/1697] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:19 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:20.070966 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b70ae-117e-4267-8987-7185a80bf334 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:20.085750 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b70ae-117e-4267-8987-7185a80bf334 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:20.335933 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243fe27-9db9-453f-a223-169ffe77548e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:20.337186 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243fe27-9db9-453f-a223-169ffe77548e admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:20.337475 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243fe27-9db9-453f-a223-169ffe77548e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:20.337755 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243fe27-9db9-453f-a223-169ffe77548e admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:20.338692 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243fe27-9db9-453f-a223-169ffe77548e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:20.343696 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243fe27-9db9-453f-a223-169ffe77548e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:20.348077 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62a3a14-75ef-426d-a862-ecca4f5900ce admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:20.364405 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d6266-fe3c-49b8-84c1-31f4823e4caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:20.365227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d6266-fe3c-49b8-84c1-31f4823e4caf admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:20.365987 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d6266-fe3c-49b8-84c1-31f4823e4caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:20.365987 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d6266-fe3c-49b8-84c1-31f4823e4caf admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:20.366781 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42d6266-fe3c-49b8-84c1-31f4823e4caf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:26:20.367093 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42d6266-fe3c-49b8-84c1-31f4823e4caf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:20.376934 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42d6266-fe3c-49b8-84c1-31f4823e4caf admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:20.403329 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 838/1698] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:26:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:26:20.404572 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d6266-fe3c-49b8-84c1-31f4823e4caf admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:20.432445 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d6266-fe3c-49b8-84c1-31f4823e4caf admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:20.447301 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a3280d-f5c5-496b-993c-884588601045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:20.449021 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3280d-f5c5-496b-993c-884588601045 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:20.455486 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3280d-f5c5-496b-993c-884588601045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:20.455486 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3280d-f5c5-496b-993c-884588601045 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:20.455486 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a3280d-f5c5-496b-993c-884588601045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:20.455486 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a3280d-f5c5-496b-993c-884588601045 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:20.490362 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 861/1699] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:26:20 2022] POST /identity/v3/projects => generated 365 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:20.490362 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243fe27-9db9-453f-a223-169ffe77548e admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:20.493456 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-8243fe27-9db9-453f-a223-169ffe77548e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:20.512527 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 839/1700] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:20 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:20.513947 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a3280d-f5c5-496b-993c-884588601045 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:20.532559 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:20.540535 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a3280d-f5c5-496b-993c-884588601045 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:20.631526 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:20.715384 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eddc3d-0ce8-43ff-a46c-65cd3fa112ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:20.716093 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eddc3d-0ce8-43ff-a46c-65cd3fa112ab admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:20.717096 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eddc3d-0ce8-43ff-a46c-65cd3fa112ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:20.717096 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eddc3d-0ce8-43ff-a46c-65cd3fa112ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:20.717364 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eddc3d-0ce8-43ff-a46c-65cd3fa112ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:20.717474 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243fe27-9db9-453f-a223-169ffe77548e admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:20.719560 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eddc3d-0ce8-43ff-a46c-65cd3fa112ab admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:20.728638 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b43f5-61ed-4932-9c3a-041364314ea4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:20.733412 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b43f5-61ed-4932-9c3a-041364314ea4 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:20.733769 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b43f5-61ed-4932-9c3a-041364314ea4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:20.734146 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b43f5-61ed-4932-9c3a-041364314ea4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:20.829058 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 840/1701] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:26:20 2022] POST /identity/v3/projects => generated 373 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:20.830314 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eddc3d-0ce8-43ff-a46c-65cd3fa112ab admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:20.877128 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b43f5-61ed-4932-9c3a-041364314ea4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:20.879278 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b43f5-61ed-4932-9c3a-041364314ea4 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:20.907419 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 862/1702] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:20 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:20.921339 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b43f5-61ed-4932-9c3a-041364314ea4 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:20.957666 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b43f5-61ed-4932-9c3a-041364314ea4 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:20.987822 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eddc3d-0ce8-43ff-a46c-65cd3fa112ab admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:21.004059 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad607e-38c9-4307-8761-f873981c5ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:21.009341 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad607e-38c9-4307-8761-f873981c5ecd admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:21.009642 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad607e-38c9-4307-8761-f873981c5ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:21.009983 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad607e-38c9-4307-8761-f873981c5ecd admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:21.010751 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bad607e-38c9-4307-8761-f873981c5ecd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:21.012932 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bad607e-38c9-4307-8761-f873981c5ecd admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:21.065139 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2cbf28-8188-49de-8915-97f5b9dc6ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:21.066166 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2cbf28-8188-49de-8915-97f5b9dc6ca9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:21.066514 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2cbf28-8188-49de-8915-97f5b9dc6ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:21.066883 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2cbf28-8188-49de-8915-97f5b9dc6ca9 admin admin] PATH_INFO: `/v3/projects/adaf93a38977429db9cfae6ee619e0a8/users/a378fd60c6af4778a443a21f5bcca46a/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:21.070614 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2cbf28-8188-49de-8915-97f5b9dc6ca9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adaf93a38977429db9cfae6ee619e0a8, user_id=a378fd60c6af4778a443a21f5bcca46a, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:21.074205 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2cbf28-8188-49de-8915-97f5b9dc6ca9 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:21.087974 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 863/1703] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:20 2022] PUT /identity/v3/projects/adaf93a38977429db9cfae6ee619e0a8/users/a378fd60c6af4778a443a21f5bcca46a/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:21.089420 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3d70d-ab15-49f5-9fa1-0cc10e4cf863 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:21.089737 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3d70d-ab15-49f5-9fa1-0cc10e4cf863 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:21.091087 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3d70d-ab15-49f5-9fa1-0cc10e4cf863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:21.126192 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-cee3d70d-ab15-49f5-9fa1-0cc10e4cf863 None None] MFA Rules not processed for user `76f3dd56c5d24c7490e337b2aa90b584`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:21.127496 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-cee3d70d-ab15-49f5-9fa1-0cc10e4cf863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:21.138327 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 841/1704] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:26:20 2022] POST /identity/v3/users => generated 283 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:21.139267 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad607e-38c9-4307-8761-f873981c5ecd admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:21.153644 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad607e-38c9-4307-8761-f873981c5ecd admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:21.258361 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:21.279897 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb2ae2e-02d2-4cbf-9526-ccdc7e90ccf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:21.286628 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2ae2e-02d2-4cbf-9526-ccdc7e90ccf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:21.287725 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2ae2e-02d2-4cbf-9526-ccdc7e90ccf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:21.288255 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2ae2e-02d2-4cbf-9526-ccdc7e90ccf6 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:21.290523 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb2ae2e-02d2-4cbf-9526-ccdc7e90ccf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:21.293720 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb2ae2e-02d2-4cbf-9526-ccdc7e90ccf6 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:21.310796 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 864/1705] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:21.312190 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7435507-e438-4f9e-ba67-2c93dea4288b None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:21.312548 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7435507-e438-4f9e-ba67-2c93dea4288b None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:21.312831 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7435507-e438-4f9e-ba67-2c93dea4288b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:21.343145 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-f7435507-e438-4f9e-ba67-2c93dea4288b None None] MFA Rules not processed for user `788e2a26fe274360aed5498ef853e013`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:21.345821 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-f7435507-e438-4f9e-ba67-2c93dea4288b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:21.413335 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 842/1706] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:26:21 2022] POST /identity/v3/users => generated 287 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:21.416973 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530391d2-e434-44d4-a517-4b57123ca307 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:21.429232 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530391d2-e434-44d4-a517-4b57123ca307 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:21.430015 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530391d2-e434-44d4-a517-4b57123ca307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:21.493548 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 865/1707] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:21.496324 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a838c0-f027-4c43-b7e0-dc53b31ffc96 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:21.497206 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a838c0-f027-4c43-b7e0-dc53b31ffc96 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:21.497827 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a838c0-f027-4c43-b7e0-dc53b31ffc96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:21.519286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-530391d2-e434-44d4-a517-4b57123ca307 None None] MFA Rules not processed for user `683c8a9abae1455bb2ec773b7608d03c`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:21.522459 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-530391d2-e434-44d4-a517-4b57123ca307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:21.554154 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-15a838c0-f027-4c43-b7e0-dc53b31ffc96 None None] MFA Rules not processed for user `a378fd60c6af4778a443a21f5bcca46a`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:21.557476 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-15a838c0-f027-4c43-b7e0-dc53b31ffc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:21.572521 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:21.672336 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 843/1708] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:21.699529 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530391d2-e434-44d4-a517-4b57123ca307 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:21.733299 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530391d2-e434-44d4-a517-4b57123ca307 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:21.735499 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 866/1709] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:21 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:21.737873 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326383c9-5fbf-4e83-886a-dfb44373c3f3 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:21.738361 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326383c9-5fbf-4e83-886a-dfb44373c3f3 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:21.741636 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326383c9-5fbf-4e83-886a-dfb44373c3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:21.747082 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5378100a-f86b-4c2d-b281-c7f195bab1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:21.747870 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378100a-f86b-4c2d-b281-c7f195bab1ef admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:21.748400 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378100a-f86b-4c2d-b281-c7f195bab1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:21.748579 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378100a-f86b-4c2d-b281-c7f195bab1ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:21.749399 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5378100a-f86b-4c2d-b281-c7f195bab1ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:21.751063 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5378100a-f86b-4c2d-b281-c7f195bab1ef admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:21.771256 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 844/1710] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:21.773850 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5378100a-f86b-4c2d-b281-c7f195bab1ef admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:21.788273 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5378100a-f86b-4c2d-b281-c7f195bab1ef admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:21.794573 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc5176-8881-48ef-91f6-95a1ed9ee090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:21.795347 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc5176-8881-48ef-91f6-95a1ed9ee090 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:21.795641 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc5176-8881-48ef-91f6-95a1ed9ee090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:21.795973 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc5176-8881-48ef-91f6-95a1ed9ee090 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:21.796776 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bc5176-8881-48ef-91f6-95a1ed9ee090 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:21.805424 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bc5176-8881-48ef-91f6-95a1ed9ee090 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:21.821115 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 845/1711] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:21.823135 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-326383c9-5fbf-4e83-886a-dfb44373c3f3 None None] MFA Rules not processed for user `76f3dd56c5d24c7490e337b2aa90b584`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:21.825179 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-326383c9-5fbf-4e83-886a-dfb44373c3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:21.829516 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414348a6-e1d1-4f40-972e-6e41666fcbc2 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:21.838989 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414348a6-e1d1-4f40-972e-6e41666fcbc2 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:21.839862 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414348a6-e1d1-4f40-972e-6e41666fcbc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:21.860221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 867/1712] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:21.862457 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326383c9-5fbf-4e83-886a-dfb44373c3f3 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:21.877543 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-326383c9-5fbf-4e83-886a-dfb44373c3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:21.894444 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-414348a6-e1d1-4f40-972e-6e41666fcbc2 None None] MFA Rules not processed for user `788e2a26fe274360aed5498ef853e013`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:21.900010 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-414348a6-e1d1-4f40-972e-6e41666fcbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:21.931321 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 846/1713] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:21.932965 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414348a6-e1d1-4f40-972e-6e41666fcbc2 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:21.949191 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414348a6-e1d1-4f40-972e-6e41666fcbc2 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:21.961084 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446658a-0416-42b5-903e-c07c68078db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:21.961900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446658a-0416-42b5-903e-c07c68078db3 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:21.962442 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446658a-0416-42b5-903e-c07c68078db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:21.963118 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446658a-0416-42b5-903e-c07c68078db3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:21.963948 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e446658a-0416-42b5-903e-c07c68078db3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:21.965918 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:21.967325 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e446658a-0416-42b5-903e-c07c68078db3 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:21.988359 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 847/1714] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:21.988359 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446658a-0416-42b5-903e-c07c68078db3 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:21.993225 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e446658a-0416-42b5-903e-c07c68078db3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:22.016989 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326383c9-5fbf-4e83-886a-dfb44373c3f3 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:22.025700 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7049e-efd3-4034-bf0c-d9ca5451b6cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:22.026710 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7049e-efd3-4034-bf0c-d9ca5451b6cb service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:22.026947 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7049e-efd3-4034-bf0c-d9ca5451b6cb service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:22.027286 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7049e-efd3-4034-bf0c-d9ca5451b6cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:22.042330 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b7049e-efd3-4034-bf0c-d9ca5451b6cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:22.044296 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b7049e-efd3-4034-bf0c-d9ca5451b6cb service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:22.061698 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 868/1715] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:22.063795 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7049e-efd3-4034-bf0c-d9ca5451b6cb service nova] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:22.076458 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7049e-efd3-4034-bf0c-d9ca5451b6cb service nova] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:22.081815 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c8023-549a-4330-b360-618e979b06c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:22.082663 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c8023-549a-4330-b360-618e979b06c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:22.083808 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c8023-549a-4330-b360-618e979b06c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:22.084250 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c8023-549a-4330-b360-618e979b06c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:22.086280 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833c8023-549a-4330-b360-618e979b06c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:22.088053 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833c8023-549a-4330-b360-618e979b06c3 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:22.104613 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 869/1716] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:22.104613 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c8023-549a-4330-b360-618e979b06c3 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:22.105037 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-833c8023-549a-4330-b360-618e979b06c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:22.144365 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c8023-549a-4330-b360-618e979b06c3 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:22.161964 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab17ad9-500d-4c7c-9ce0-c3046c2ccf66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:22.166572 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab17ad9-500d-4c7c-9ce0-c3046c2ccf66 service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:22.166572 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab17ad9-500d-4c7c-9ce0-c3046c2ccf66 service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:22.166572 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab17ad9-500d-4c7c-9ce0-c3046c2ccf66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:22.190574 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab17ad9-500d-4c7c-9ce0-c3046c2ccf66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:22.193325 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab17ad9-500d-4c7c-9ce0-c3046c2ccf66 service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:22.198797 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446658a-0416-42b5-903e-c07c68078db3 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:22.214718 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b770280-d8bc-431e-8dc6-4d101a6ee913 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:22.219607 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 870/1717] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:22.220785 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b770280-d8bc-431e-8dc6-4d101a6ee913 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:22.222706 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab17ad9-500d-4c7c-9ce0-c3046c2ccf66 service nova] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:22.225656 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b770280-d8bc-431e-8dc6-4d101a6ee913 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:22.226521 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b770280-d8bc-431e-8dc6-4d101a6ee913 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:22.247161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab17ad9-500d-4c7c-9ce0-c3046c2ccf66 service nova] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:22.254153 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1946f4-7ce8-403d-b47c-02a00ee3814d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:22.262357 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1946f4-7ce8-403d-b47c-02a00ee3814d admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:22.263147 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1946f4-7ce8-403d-b47c-02a00ee3814d admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:22.263778 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1946f4-7ce8-403d-b47c-02a00ee3814d admin admin] PATH_INFO: `/v3/projects/3f3b2caf25394630b17da048591d2fe8/users/f3ce2bd4b7da434faa9bbd4563042d25/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:22.299179 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b770280-d8bc-431e-8dc6-4d101a6ee913 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:22.300923 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b770280-d8bc-431e-8dc6-4d101a6ee913 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:22.302861 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1946f4-7ce8-403d-b47c-02a00ee3814d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f3b2caf25394630b17da048591d2fe8, user_id=f3ce2bd4b7da434faa9bbd4563042d25, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:22.306039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1946f4-7ce8-403d-b47c-02a00ee3814d admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:22.315460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 848/1718] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:21 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:22.318417 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b770280-d8bc-431e-8dc6-4d101a6ee913 service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:22.338585 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b770280-d8bc-431e-8dc6-4d101a6ee913 service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:22.346159 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c533d3-180d-4ce5-98b0-979dee8b052b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:22.348029 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c533d3-180d-4ce5-98b0-979dee8b052b admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:22.348029 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c533d3-180d-4ce5-98b0-979dee8b052b admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:22.349842 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c533d3-180d-4ce5-98b0-979dee8b052b admin admin] PATH_INFO: `/v3/projects/ddee550fc53841359ae9a46735298334/users/4bea4bd0562a4df390a522e24bb9bd55/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:22.387248 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c533d3-180d-4ce5-98b0-979dee8b052b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddee550fc53841359ae9a46735298334, user_id=4bea4bd0562a4df390a522e24bb9bd55, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:22.391328 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c533d3-180d-4ce5-98b0-979dee8b052b admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:22.417940 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 871/1719] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:22 2022] PUT /identity/v3/projects/3f3b2caf25394630b17da048591d2fe8/users/f3ce2bd4b7da434faa9bbd4563042d25/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:22.423979 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1946f4-7ce8-403d-b47c-02a00ee3814d admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:22.540114 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 849/1720] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:22 2022] PUT /identity/v3/projects/ddee550fc53841359ae9a46735298334/users/4bea4bd0562a4df390a522e24bb9bd55/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:22.546184 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159faae1-6275-4ac0-b545-d12b27782833 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:22.549970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159faae1-6275-4ac0-b545-d12b27782833 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:22.549970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159faae1-6275-4ac0-b545-d12b27782833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:22.588650 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1946f4-7ce8-403d-b47c-02a00ee3814d admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:22.661941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-159faae1-6275-4ac0-b545-d12b27782833 None None] MFA Rules not processed for user `f3ce2bd4b7da434faa9bbd4563042d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:22.668067 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-159faae1-6275-4ac0-b545-d12b27782833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:22.917146 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd824097-2084-4187-9a63-12a09e8f8a2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:22.919574 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd824097-2084-4187-9a63-12a09e8f8a2d service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:22.921042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd824097-2084-4187-9a63-12a09e8f8a2d service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:22.923714 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd824097-2084-4187-9a63-12a09e8f8a2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:22.926719 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:22.963247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 850/1721] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:22 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:22.981859 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2365df-f7b9-4070-a811-bbfe4d8a110c None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:22.982502 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2365df-f7b9-4070-a811-bbfe4d8a110c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:22.983197 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2365df-f7b9-4070-a811-bbfe4d8a110c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:23.024873 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:23.119199 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-7d2365df-f7b9-4070-a811-bbfe4d8a110c None None] MFA Rules not processed for user `4bea4bd0562a4df390a522e24bb9bd55`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:23.122763 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7d2365df-f7b9-4070-a811-bbfe4d8a110c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:23.256105 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd824097-2084-4187-9a63-12a09e8f8a2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:23.259007 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd824097-2084-4187-9a63-12a09e8f8a2d service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:23.315201 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 872/1722] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 887 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:23.315201 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd824097-2084-4187-9a63-12a09e8f8a2d service nova] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:23.347061 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 851/1723] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:23.352565 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2365df-f7b9-4070-a811-bbfe4d8a110c None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:23.501057 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd824097-2084-4187-9a63-12a09e8f8a2d service nova] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:23.510732 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f8347-d74d-4885-856f-cfdf9f6b4899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:23.511395 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f8347-d74d-4885-856f-cfdf9f6b4899 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:23.511683 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f8347-d74d-4885-856f-cfdf9f6b4899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:23.512016 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f8347-d74d-4885-856f-cfdf9f6b4899 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:23.512792 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0f8347-d74d-4885-856f-cfdf9f6b4899 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:23.514850 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0f8347-d74d-4885-856f-cfdf9f6b4899 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:23.542329 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2365df-f7b9-4070-a811-bbfe4d8a110c None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:23.553618 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ca8bca-f6bb-4aa9-a000-795af991fef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:23.554895 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca8bca-f6bb-4aa9-a000-795af991fef2 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:23.555900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca8bca-f6bb-4aa9-a000-795af991fef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:23.556681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca8bca-f6bb-4aa9-a000-795af991fef2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:23.558278 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ca8bca-f6bb-4aa9-a000-795af991fef2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:23.562463 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ca8bca-f6bb-4aa9-a000-795af991fef2 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:23.732639 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 873/1724] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:26:23 2022] POST /identity/v3/projects => generated 365 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:23.749488 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72863c47-3a98-4a9a-82bb-09f8fb888fd9 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:23.750434 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72863c47-3a98-4a9a-82bb-09f8fb888fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:23.750434 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72863c47-3a98-4a9a-82bb-09f8fb888fd9 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:23.752951 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 874/1725] 10.222.0.40 () {54 vars in 912 bytes} [Mon Jul 18 06:26:23 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:26:23.759530 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72863c47-3a98-4a9a-82bb-09f8fb888fd9 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:23.775242 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 852/1726] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:26:23 2022] POST /identity/v3/projects => generated 373 bytes in 427 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:23.778685 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ca8bca-f6bb-4aa9-a000-795af991fef2 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:23.910181 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72863c47-3a98-4a9a-82bb-09f8fb888fd9 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:23.938802 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ca8bca-f6bb-4aa9-a000-795af991fef2 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:23.944704 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6af0a-d684-4169-9c15-e7b1b5fe3601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:23.945590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6af0a-d684-4169-9c15-e7b1b5fe3601 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:23.945980 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6af0a-d684-4169-9c15-e7b1b5fe3601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:23.946368 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6af0a-d684-4169-9c15-e7b1b5fe3601 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:23.947195 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb6af0a-d684-4169-9c15-e7b1b5fe3601 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:23.950919 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb6af0a-d684-4169-9c15-e7b1b5fe3601 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:24.041043 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:24.053414 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0079ee55-ff56-4305-8e6d-489896dbfb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:24.054846 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079ee55-ff56-4305-8e6d-489896dbfb2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:24.055156 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079ee55-ff56-4305-8e6d-489896dbfb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:24.055516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079ee55-ff56-4305-8e6d-489896dbfb2b admin admin] PATH_INFO: `/v3/users/d32b8a126e744671a81a63a9cb73ae21` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:24.056179 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:24.057960 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0079ee55-ff56-4305-8e6d-489896dbfb2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d32b8a126e744671a81a63a9cb73ae21)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:24.062341 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0079ee55-ff56-4305-8e6d-489896dbfb2b admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:24.074531 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 853/1727] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:26:23 2022] POST /identity/v3/users => generated 289 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:24.074531 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65dd2f6-5a32-492c-9578-0556093a264c None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:24.074531 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65dd2f6-5a32-492c-9578-0556093a264c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:24.075178 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65dd2f6-5a32-492c-9578-0556093a264c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:24.194949 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-e65dd2f6-5a32-492c-9578-0556093a264c None None] MFA Rules not processed for user `9824a96e2bcd4a16bfc6fe6bf4ea1ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:24.196624 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e65dd2f6-5a32-492c-9578-0556093a264c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:24.282376 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 854/1728] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Jul 18 06:26:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:24.283488 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65dd2f6-5a32-492c-9578-0556093a264c None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:24.307046 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65dd2f6-5a32-492c-9578-0556093a264c None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:24.313021 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bbdbdc-9829-44c9-9f26-4e2acf802786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:24.313729 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bbdbdc-9829-44c9-9f26-4e2acf802786 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:24.314022 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bbdbdc-9829-44c9-9f26-4e2acf802786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:24.314305 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bbdbdc-9829-44c9-9f26-4e2acf802786 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:24.315726 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bbdbdc-9829-44c9-9f26-4e2acf802786 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:24.319542 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bbdbdc-9829-44c9-9f26-4e2acf802786 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:24.344250 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0079ee55-ff56-4305-8e6d-489896dbfb2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd32b8a126e744671a81a63a9cb73ae21'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:24.354643 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0079ee55-ff56-4305-8e6d-489896dbfb2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd32b8a126e744671a81a63a9cb73ae21'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:24.356105 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0079ee55-ff56-4305-8e6d-489896dbfb2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd32b8a126e744671a81a63a9cb73ae21'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:24.357085 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0079ee55-ff56-4305-8e6d-489896dbfb2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd32b8a126e744671a81a63a9cb73ae21'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:24.378395 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0079ee55-ff56-4305-8e6d-489896dbfb2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd32b8a126e744671a81a63a9cb73ae21'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:24.415847 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 855/1729] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:26:24 2022] POST /identity/v3/users => generated 293 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:24.417878 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c65ce6-5eb2-42e0-8003-46819b849f96 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:24.418149 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c65ce6-5eb2-42e0-8003-46819b849f96 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:24.419910 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c65ce6-5eb2-42e0-8003-46819b849f96 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:24.445478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 856/1730] 10.222.0.40 () {54 vars in 912 bytes} [Mon Jul 18 06:26:24 2022] GET /identity => generated 270 bytes in 26 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:26:24.447610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c65ce6-5eb2-42e0-8003-46819b849f96 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:24.453952 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-91c65ce6-5eb2-42e0-8003-46819b849f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:24.513141 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 875/1731] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:26:23 2022] DELETE /identity/v3/users/d32b8a126e744671a81a63a9cb73ae21 => generated 0 bytes in 759 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:24.514193 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0079ee55-ff56-4305-8e6d-489896dbfb2b admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:24.519250 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c65ce6-5eb2-42e0-8003-46819b849f96 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:24.527830 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-0079ee55-ff56-4305-8e6d-489896dbfb2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:24.635556 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a472429-cc22-432b-8a1b-dee1b6eef6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:24.637279 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a472429-cc22-432b-8a1b-dee1b6eef6a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:24.652708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a472429-cc22-432b-8a1b-dee1b6eef6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:24.652708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a472429-cc22-432b-8a1b-dee1b6eef6a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:24.652708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a472429-cc22-432b-8a1b-dee1b6eef6a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:24.652708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a472429-cc22-432b-8a1b-dee1b6eef6a3 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:24.674420 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 857/1732] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:24.674643 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0079ee55-ff56-4305-8e6d-489896dbfb2b admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:24.674741 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a472429-cc22-432b-8a1b-dee1b6eef6a3 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:24.680563 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7a472429-cc22-432b-8a1b-dee1b6eef6a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:24.685603 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4f5cb-4d87-47e8-828f-2980a810bea2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:24.686938 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4f5cb-4d87-47e8-828f-2980a810bea2 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:24.687213 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4f5cb-4d87-47e8-828f-2980a810bea2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:24.687500 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4f5cb-4d87-47e8-828f-2980a810bea2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:24.690070 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-05f4f5cb-4d87-47e8-828f-2980a810bea2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:24.704813 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f4f5cb-4d87-47e8-828f-2980a810bea2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:24.706366 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f4f5cb-4d87-47e8-828f-2980a810bea2 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:24.712451 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a472429-cc22-432b-8a1b-dee1b6eef6a3 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:24.718704 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b54943-132f-4877-a33c-d769e1187d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:24.719455 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b54943-132f-4877-a33c-d769e1187d8c admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:24.719747 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b54943-132f-4877-a33c-d769e1187d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:24.720029 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b54943-132f-4877-a33c-d769e1187d8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:24.720851 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b54943-132f-4877-a33c-d769e1187d8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:24.722662 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b54943-132f-4877-a33c-d769e1187d8c admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:24.733630 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 876/1733] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:24.733630 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a00b2-15d5-4643-96ec-75ab240d1a86 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:24.733630 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a00b2-15d5-4643-96ec-75ab240d1a86 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:24.733630 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a00b2-15d5-4643-96ec-75ab240d1a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:24.741756 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 858/1734] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:24.743203 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ec0b5-2463-400e-9def-5347bd4a14b1 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:24.743613 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ec0b5-2463-400e-9def-5347bd4a14b1 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:24.743904 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ec0b5-2463-400e-9def-5347bd4a14b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:24.770485 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-a86ec0b5-2463-400e-9def-5347bd4a14b1 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:24.771927 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a86ec0b5-2463-400e-9def-5347bd4a14b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:24.789981 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 859/1735] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:24.796131 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ec0b5-2463-400e-9def-5347bd4a14b1 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:24.800777 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a86ec0b5-2463-400e-9def-5347bd4a14b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:24.827660 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-1d2a00b2-15d5-4643-96ec-75ab240d1a86 None None] MFA Rules not processed for user `9824a96e2bcd4a16bfc6fe6bf4ea1ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:24.827660 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-1d2a00b2-15d5-4643-96ec-75ab240d1a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:24.846210 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 877/1736] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Jul 18 06:26:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:24.847450 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a00b2-15d5-4643-96ec-75ab240d1a86 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:24.867786 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a00b2-15d5-4643-96ec-75ab240d1a86 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:24.874759 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50219614-9e55-4f48-a3f8-34cbf4f3d70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:24.877005 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50219614-9e55-4f48-a3f8-34cbf4f3d70c admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:24.877441 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50219614-9e55-4f48-a3f8-34cbf4f3d70c admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:24.877818 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50219614-9e55-4f48-a3f8-34cbf4f3d70c admin admin] PATH_INFO: `/v3/projects/1a86a5020fe34e098377d2017b59eace/users/7128398b42f64c64b3b69357133230a4/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:24.947899 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50219614-9e55-4f48-a3f8-34cbf4f3d70c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a86a5020fe34e098377d2017b59eace, user_id=7128398b42f64c64b3b69357133230a4, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:24.953523 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50219614-9e55-4f48-a3f8-34cbf4f3d70c admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:24.986090 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:24.999914 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ec0b5-2463-400e-9def-5347bd4a14b1 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:25.021775 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c2290-c578-40cf-a6d8-8d0195a9dbba tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a378fd60c6af4778a443a21f5bcca46a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adaf93a38977429db9cfae6ee619e0a8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:25.022577 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c2290-c578-40cf-a6d8-8d0195a9dbba tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:25.022941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c2290-c578-40cf-a6d8-8d0195a9dbba tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:25.023706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c2290-c578-40cf-a6d8-8d0195a9dbba tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:25.024915 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c2290-c578-40cf-a6d8-8d0195a9dbba tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:26:25.029968 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c2290-c578-40cf-a6d8-8d0195a9dbba tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:25.031733 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c2290-c578-40cf-a6d8-8d0195a9dbba tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:25.051419 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 878/1737] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:24 2022] PUT /identity/v3/projects/1a86a5020fe34e098377d2017b59eace/users/7128398b42f64c64b3b69357133230a4/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:25.053386 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50219614-9e55-4f48-a3f8-34cbf4f3d70c admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:25.065223 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 860/1738] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:26:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:26:25.066125 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c2290-c578-40cf-a6d8-8d0195a9dbba tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:25.080472 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-040c2290-c578-40cf-a6d8-8d0195a9dbba tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:25.113369 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c2290-c578-40cf-a6d8-8d0195a9dbba tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:25.197249 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ede9fe-dc67-477a-9550-0273adcc60e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:25.201908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ede9fe-dc67-477a-9550-0273adcc60e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:25.201908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ede9fe-dc67-477a-9550-0273adcc60e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:25.201908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ede9fe-dc67-477a-9550-0273adcc60e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:25.202713 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:25.263784 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50219614-9e55-4f48-a3f8-34cbf4f3d70c admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:25.285499 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2fc94-5866-4ebd-ae79-f69b18fecc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:25.288039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2fc94-5866-4ebd-ae79-f69b18fecc78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:25.289365 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2fc94-5866-4ebd-ae79-f69b18fecc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:25.290301 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2fc94-5866-4ebd-ae79-f69b18fecc78 admin admin] PATH_INFO: `/v3/projects/4cb86cac3344490d8df5fdd964a2df27/users/96411fff4f144b70b46b21670cb6f9ac/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:25.308125 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ede9fe-dc67-477a-9550-0273adcc60e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:25.310292 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ede9fe-dc67-477a-9550-0273adcc60e9 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:25.327688 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 861/1739] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:25.331226 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ede9fe-dc67-477a-9550-0273adcc60e9 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:25.331377 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e2fc94-5866-4ebd-ae79-f69b18fecc78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cb86cac3344490d8df5fdd964a2df27, user_id=96411fff4f144b70b46b21670cb6f9ac, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:25.334775 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e2fc94-5866-4ebd-ae79-f69b18fecc78 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:25.342656 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ede9fe-dc67-477a-9550-0273adcc60e9 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:25.354104 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7174c85-bb36-4b2e-afec-a278d0103f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:25.354771 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7174c85-bb36-4b2e-afec-a278d0103f7f admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:25.355068 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7174c85-bb36-4b2e-afec-a278d0103f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:25.355444 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7174c85-bb36-4b2e-afec-a278d0103f7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:25.358041 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7174c85-bb36-4b2e-afec-a278d0103f7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:25.363534 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7174c85-bb36-4b2e-afec-a278d0103f7f admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:25.392349 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 862/1740] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:25.394495 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7174c85-bb36-4b2e-afec-a278d0103f7f admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:25.440670 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 879/1741] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:25 2022] PUT /identity/v3/projects/4cb86cac3344490d8df5fdd964a2df27/users/96411fff4f144b70b46b21670cb6f9ac/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:25.446040 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2fc94-5866-4ebd-ae79-f69b18fecc78 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:25.577875 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2fc94-5866-4ebd-ae79-f69b18fecc78 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:25.581829 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257007f-54ee-4155-91d1-7cf288197f84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:25.582518 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257007f-54ee-4155-91d1-7cf288197f84 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:25.583151 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257007f-54ee-4155-91d1-7cf288197f84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:25.583151 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257007f-54ee-4155-91d1-7cf288197f84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:25.598374 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8257007f-54ee-4155-91d1-7cf288197f84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:25.602218 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8257007f-54ee-4155-91d1-7cf288197f84 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:25.605334 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7174c85-bb36-4b2e-afec-a278d0103f7f admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:25.613376 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 880/1742] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:25.615011 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257007f-54ee-4155-91d1-7cf288197f84 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:25.616635 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-8257007f-54ee-4155-91d1-7cf288197f84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:25.711021 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:25.762791 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ca6ea-d76b-4ab9-8cbb-d4e961580dbd tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a378fd60c6af4778a443a21f5bcca46a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adaf93a38977429db9cfae6ee619e0a8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:25.764845 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ca6ea-d76b-4ab9-8cbb-d4e961580dbd tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:25.776333 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ca6ea-d76b-4ab9-8cbb-d4e961580dbd tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:25.776830 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ca6ea-d76b-4ab9-8cbb-d4e961580dbd tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:25.778926 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769ca6ea-d76b-4ab9-8cbb-d4e961580dbd tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:25.781910 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769ca6ea-d76b-4ab9-8cbb-d4e961580dbd tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:25.795691 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257007f-54ee-4155-91d1-7cf288197f84 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:25.838587 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d49632-d23a-4cf4-aebc-8c1b7e4a633f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:25.845292 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d49632-d23a-4cf4-aebc-8c1b7e4a633f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:25.845292 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d49632-d23a-4cf4-aebc-8c1b7e4a633f admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:25.845292 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d49632-d23a-4cf4-aebc-8c1b7e4a633f admin admin] PATH_INFO: `/v3/users/d35fb3e36d7a46a0ab79e23eb2286bb7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:25.845292 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d49632-d23a-4cf4-aebc-8c1b7e4a633f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d35fb3e36d7a46a0ab79e23eb2286bb7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:25.845292 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d49632-d23a-4cf4-aebc-8c1b7e4a633f admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:25.948102 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 863/1743] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:26:25 2022] POST /identity/v3/projects => generated 355 bytes in 555 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:25.950058 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ca6ea-d76b-4ab9-8cbb-d4e961580dbd tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:26.118271 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ca6ea-d76b-4ab9-8cbb-d4e961580dbd tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:26.125497 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ebcb56-9fe1-4204-bdc1-df74f59f3547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:26.127691 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ebcb56-9fe1-4204-bdc1-df74f59f3547 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:26.128093 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ebcb56-9fe1-4204-bdc1-df74f59f3547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:26.128387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ebcb56-9fe1-4204-bdc1-df74f59f3547 admin admin] PATH_INFO: `/v3/projects/1a86a5020fe34e098377d2017b59eace/users/7128398b42f64c64b3b69357133230a4/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:26.134257 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ebcb56-9fe1-4204-bdc1-df74f59f3547 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a86a5020fe34e098377d2017b59eace, user_id=7128398b42f64c64b3b69357133230a4, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:26.146014 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ebcb56-9fe1-4204-bdc1-df74f59f3547 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:26.150544 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d9d49632-d23a-4cf4-aebc-8c1b7e4a633f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd35fb3e36d7a46a0ab79e23eb2286bb7'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:26.174978 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d9d49632-d23a-4cf4-aebc-8c1b7e4a633f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd35fb3e36d7a46a0ab79e23eb2286bb7'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:26.174978 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d9d49632-d23a-4cf4-aebc-8c1b7e4a633f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd35fb3e36d7a46a0ab79e23eb2286bb7'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:26.174978 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d9d49632-d23a-4cf4-aebc-8c1b7e4a633f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd35fb3e36d7a46a0ab79e23eb2286bb7'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:26.181403 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 864/1744] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:25 2022] PUT /identity/v3/projects/1a86a5020fe34e098377d2017b59eace/users/7128398b42f64c64b3b69357133230a4/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:26.182376 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ebcb56-9fe1-4204-bdc1-df74f59f3547 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:26.193000 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f5ebcb56-9fe1-4204-bdc1-df74f59f3547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:26.206143 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:26.210058 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d9d49632-d23a-4cf4-aebc-8c1b7e4a633f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd35fb3e36d7a46a0ab79e23eb2286bb7'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:26.314745 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ebcb56-9fe1-4204-bdc1-df74f59f3547 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:26.335852 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bc0e4a-cec7-42f6-a5b4-2ecaaa7f6e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:26.338884 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc0e4a-cec7-42f6-a5b4-2ecaaa7f6e18 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:26.338884 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc0e4a-cec7-42f6-a5b4-2ecaaa7f6e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:26.338884 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc0e4a-cec7-42f6-a5b4-2ecaaa7f6e18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:26.338884 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bc0e4a-cec7-42f6-a5b4-2ecaaa7f6e18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:26.339513 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 881/1745] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:26:25 2022] DELETE /identity/v3/users/d35fb3e36d7a46a0ab79e23eb2286bb7 => generated 0 bytes in 726 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:26.339624 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bc0e4a-cec7-42f6-a5b4-2ecaaa7f6e18 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:26.340392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d49632-d23a-4cf4-aebc-8c1b7e4a633f admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:26.346814 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d9d49632-d23a-4cf4-aebc-8c1b7e4a633f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:26.362646 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 865/1746] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:26.364633 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e6310-b93e-43d5-84cc-a012fcb30dbd None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:26.365099 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e6310-b93e-43d5-84cc-a012fcb30dbd None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:26.365425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e6310-b93e-43d5-84cc-a012fcb30dbd None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:26.367193 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 866/1747] 10.222.0.40 () {54 vars in 961 bytes} [Mon Jul 18 06:26:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:26:26.375655 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3727c9-e2cb-498c-91ac-a069a4208297 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:26.375996 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3727c9-e2cb-498c-91ac-a069a4208297 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:26.376216 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3727c9-e2cb-498c-91ac-a069a4208297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:26.443938 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-7d3727c9-e2cb-498c-91ac-a069a4208297 None None] MFA Rules not processed for user `7128398b42f64c64b3b69357133230a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:26.445288 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7d3727c9-e2cb-498c-91ac-a069a4208297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:26.533911 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d49632-d23a-4cf4-aebc-8c1b7e4a633f admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:26.541252 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c72fa-2be6-48e1-b0d3-69d56f3929dc tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a378fd60c6af4778a443a21f5bcca46a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adaf93a38977429db9cfae6ee619e0a8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:26.542594 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c72fa-2be6-48e1-b0d3-69d56f3929dc tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:26.543111 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c72fa-2be6-48e1-b0d3-69d56f3929dc tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:26.543639 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c72fa-2be6-48e1-b0d3-69d56f3929dc tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:26.545639 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c72fa-2be6-48e1-b0d3-69d56f3929dc tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:26:26.546484 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c72fa-2be6-48e1-b0d3-69d56f3929dc tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:26.548492 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c72fa-2be6-48e1-b0d3-69d56f3929dc tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:26.566568 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:26.572525 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 882/1748] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:26:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:26:26.575054 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010520b-4be9-46fd-8c63-461df77961c6 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:26.577334 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010520b-4be9-46fd-8c63-461df77961c6 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:26.577993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010520b-4be9-46fd-8c63-461df77961c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:26.619813 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-f010520b-4be9-46fd-8c63-461df77961c6 None None] MFA Rules not processed for user `76f3dd56c5d24c7490e337b2aa90b584`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:26.622275 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-f010520b-4be9-46fd-8c63-461df77961c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:26.694848 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 867/1749] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:26 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:26.694848 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67790a16-7185-48d4-9362-f9da4cb259ce None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:26.694848 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67790a16-7185-48d4-9362-f9da4cb259ce None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:26.695592 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67790a16-7185-48d4-9362-f9da4cb259ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:26.744777 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-67790a16-7185-48d4-9362-f9da4cb259ce None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:26.747881 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-67790a16-7185-48d4-9362-f9da4cb259ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:26.887396 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 883/1750] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:26.892885 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca2748-c293-40de-abe1-f9729c0e6fb1 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:26.893624 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca2748-c293-40de-abe1-f9729c0e6fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:26.894114 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca2748-c293-40de-abe1-f9729c0e6fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:26.968662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-3eca2748-c293-40de-abe1-f9729c0e6fb1 None None] MFA Rules not processed for user `a3560534427544cdabfd0ba702a0a165`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:26.969976 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-3eca2748-c293-40de-abe1-f9729c0e6fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:27.017387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 868/1751] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:27.018649 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67790a16-7185-48d4-9362-f9da4cb259ce None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:27.080205 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67790a16-7185-48d4-9362-f9da4cb259ce None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:27.090721 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e9ee5-e5c2-446c-a35a-fb7fce324d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:27.096066 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e9ee5-e5c2-446c-a35a-fb7fce324d8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:27.109086 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e9ee5-e5c2-446c-a35a-fb7fce324d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:27.110934 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e9ee5-e5c2-446c-a35a-fb7fce324d8c admin admin] PATH_INFO: `/v3/projects/4cb86cac3344490d8df5fdd964a2df27/users/96411fff4f144b70b46b21670cb6f9ac/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:27.114010 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e9ee5-e5c2-446c-a35a-fb7fce324d8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cb86cac3344490d8df5fdd964a2df27, user_id=96411fff4f144b70b46b21670cb6f9ac, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:27.120266 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e9ee5-e5c2-446c-a35a-fb7fce324d8c admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:27.151851 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 884/1752] 10.222.0.40 () {58 vars in 1058 bytes} [Mon Jul 18 06:26:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:27.153115 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca2748-c293-40de-abe1-f9729c0e6fb1 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:27.184608 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca2748-c293-40de-abe1-f9729c0e6fb1 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:27.201429 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 869/1753] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:27 2022] PUT /identity/v3/projects/4cb86cac3344490d8df5fdd964a2df27/users/96411fff4f144b70b46b21670cb6f9ac/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:27.204723 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e9ee5-e5c2-446c-a35a-fb7fce324d8c admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:27.218578 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-9b0e9ee5-e5c2-446c-a35a-fb7fce324d8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:27.234577 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:27.331618 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a32b59-bc9c-472f-b718-6f93bb198a0e tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a378fd60c6af4778a443a21f5bcca46a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adaf93a38977429db9cfae6ee619e0a8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:27.336019 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a32b59-bc9c-472f-b718-6f93bb198a0e tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:27.336374 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a32b59-bc9c-472f-b718-6f93bb198a0e tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:27.337746 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a32b59-bc9c-472f-b718-6f93bb198a0e tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] PATH_INFO: `/v3/projects/542e9d4974c945b8b6e903c3dc6c1d86` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:27.339354 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a32b59-bc9c-472f-b718-6f93bb198a0e tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] RBAC: Authorizing `identity:delete_project(project_id=542e9d4974c945b8b6e903c3dc6c1d86)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:27.344677 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a32b59-bc9c-472f-b718-6f93bb198a0e tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:27.400806 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e9ee5-e5c2-446c-a35a-fb7fce324d8c admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:27.412748 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5ab3e-f5be-4c68-8327-30a63b4d6a23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:27.413584 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5ab3e-f5be-4c68-8327-30a63b4d6a23 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:27.413877 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5ab3e-f5be-4c68-8327-30a63b4d6a23 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:27.414159 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5ab3e-f5be-4c68-8327-30a63b4d6a23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:27.639419 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-05a32b59-bc9c-472f-b718-6f93bb198a0e tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '542e9d4974c945b8b6e903c3dc6c1d86', 'request_id': 'req-05a32b59-bc9c-472f-b718-6f93bb198a0e'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:27.665603 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-05a32b59-bc9c-472f-b718-6f93bb198a0e tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '542e9d4974c945b8b6e903c3dc6c1d86', 'request_id': 'req-05a32b59-bc9c-472f-b718-6f93bb198a0e'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:27.676113 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa5ab3e-f5be-4c68-8327-30a63b4d6a23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:27.683175 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa5ab3e-f5be-4c68-8327-30a63b4d6a23 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:27.713366 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 870/1754] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:27 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:27.714176 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5ab3e-f5be-4c68-8327-30a63b4d6a23 service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:27.715627 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-5aa5ab3e-f5be-4c68-8327-30a63b4d6a23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:27.742844 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-05a32b59-bc9c-472f-b718-6f93bb198a0e tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '542e9d4974c945b8b6e903c3dc6c1d86', 'request_id': 'req-05a32b59-bc9c-472f-b718-6f93bb198a0e'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:27.764195 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-05a32b59-bc9c-472f-b718-6f93bb198a0e tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] The token cache is being invalidate because project 542e9d4974c945b8b6e903c3dc6c1d86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:26:27.764557 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-05a32b59-bc9c-472f-b718-6f93bb198a0e tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:27.776250 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-05a32b59-bc9c-472f-b718-6f93bb198a0e tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:27.777231 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:27.785736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 885/1755] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:26:27 2022] DELETE /identity/v3/projects/542e9d4974c945b8b6e903c3dc6c1d86 => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:27.789923 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a32b59-bc9c-472f-b718-6f93bb198a0e tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:27.791751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-05a32b59-bc9c-472f-b718-6f93bb198a0e tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:27.999547 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a32b59-bc9c-472f-b718-6f93bb198a0e tempest-VolumeQuotaClassesTest-1550756753 tempest-VolumeQuotaClassesTest-1550756753-project-admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:28.016133 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8341ead-e95c-4dcd-a295-fad839df86d4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3560534427544cdabfd0ba702a0a165', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:28.039523 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8341ead-e95c-4dcd-a295-fad839df86d4 service placement] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:28.039523 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8341ead-e95c-4dcd-a295-fad839df86d4 service placement] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:28.039523 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8341ead-e95c-4dcd-a295-fad839df86d4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:28.039523 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c8341ead-e95c-4dcd-a295-fad839df86d4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:28.063358 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8341ead-e95c-4dcd-a295-fad839df86d4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:28.065191 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8341ead-e95c-4dcd-a295-fad839df86d4 service placement] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:28.091879 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 886/1756] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:28.093068 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8341ead-e95c-4dcd-a295-fad839df86d4 service placement] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:28.096904 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c8341ead-e95c-4dcd-a295-fad839df86d4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:28.197060 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5ab3e-f5be-4c68-8327-30a63b4d6a23 service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:28.284268 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:28.346718 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8341ead-e95c-4dcd-a295-fad839df86d4 service placement] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:28.370624 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8b26fc-d016-4b8f-8dd1-3e9ced80216b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:28.372108 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8b26fc-d016-4b8f-8dd1-3e9ced80216b service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:28.376876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8b26fc-d016-4b8f-8dd1-3e9ced80216b service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:28.381108 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8b26fc-d016-4b8f-8dd1-3e9ced80216b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:28.388732 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ab8b26fc-d016-4b8f-8dd1-3e9ced80216b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:28.419982 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ad81e5-2b2a-421f-825a-fe5e3496a608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:28.421316 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad81e5-2b2a-421f-825a-fe5e3496a608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:28.425015 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad81e5-2b2a-421f-825a-fe5e3496a608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:28.425579 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad81e5-2b2a-421f-825a-fe5e3496a608 admin admin] PATH_INFO: `/v3/projects/41c528319bab4e1ea147f98b548142ff` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:28.431387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ad81e5-2b2a-421f-825a-fe5e3496a608 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41c528319bab4e1ea147f98b548142ff)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:28.441197 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ad81e5-2b2a-421f-825a-fe5e3496a608 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:28.553398 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8b26fc-d016-4b8f-8dd1-3e9ced80216b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:28.555469 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8b26fc-d016-4b8f-8dd1-3e9ced80216b service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:28.571006 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 887/1757] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:28.572123 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8b26fc-d016-4b8f-8dd1-3e9ced80216b service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:28.573856 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ab8b26fc-d016-4b8f-8dd1-3e9ced80216b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:28.691343 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8b26fc-d016-4b8f-8dd1-3e9ced80216b service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:28.703119 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-65ad81e5-2b2a-421f-825a-fe5e3496a608 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41c528319bab4e1ea147f98b548142ff', 'request_id': 'req-65ad81e5-2b2a-421f-825a-fe5e3496a608'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:28.753566 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-65ad81e5-2b2a-421f-825a-fe5e3496a608 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41c528319bab4e1ea147f98b548142ff', 'request_id': 'req-65ad81e5-2b2a-421f-825a-fe5e3496a608'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:28.831825 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38af2976-aa33-482f-b365-0941ea23c8ff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3560534427544cdabfd0ba702a0a165', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:28.835302 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af2976-aa33-482f-b365-0941ea23c8ff service placement] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:28.835302 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af2976-aa33-482f-b365-0941ea23c8ff service placement] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:28.835302 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af2976-aa33-482f-b365-0941ea23c8ff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:28.851810 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38af2976-aa33-482f-b365-0941ea23c8ff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:28.853555 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38af2976-aa33-482f-b365-0941ea23c8ff service placement] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:28.858769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-65ad81e5-2b2a-421f-825a-fe5e3496a608 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41c528319bab4e1ea147f98b548142ff', 'request_id': 'req-65ad81e5-2b2a-421f-825a-fe5e3496a608'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:28.877038 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 888/1758] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:28.878722 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f68e959-a86e-4896-9d3e-40f2231a1e97 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:28.879115 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f68e959-a86e-4896-9d3e-40f2231a1e97 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:28.880018 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f68e959-a86e-4896-9d3e-40f2231a1e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:28.908562 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-65ad81e5-2b2a-421f-825a-fe5e3496a608 admin admin] The token cache is being invalidate because project 41c528319bab4e1ea147f98b548142ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:26:28.908960 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-65ad81e5-2b2a-421f-825a-fe5e3496a608 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:28.917295 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-65ad81e5-2b2a-421f-825a-fe5e3496a608 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:28.920853 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-4f68e959-a86e-4896-9d3e-40f2231a1e97 None None] MFA Rules not processed for user `96411fff4f144b70b46b21670cb6f9ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:28.921909 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 871/1759] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:26:27 2022] DELETE /identity/v3/projects/41c528319bab4e1ea147f98b548142ff => generated 0 bytes in 1208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:28.922864 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4f68e959-a86e-4896-9d3e-40f2231a1e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:28.923480 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923745fc-d7eb-4a95-b1e0-5277836ce0fb None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:28.925007 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923745fc-d7eb-4a95-b1e0-5277836ce0fb None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:28.925328 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923745fc-d7eb-4a95-b1e0-5277836ce0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:28.982429 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-923745fc-d7eb-4a95-b1e0-5277836ce0fb None None] MFA Rules not processed for user `683c8a9abae1455bb2ec773b7608d03c`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:28.988202 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-923745fc-d7eb-4a95-b1e0-5277836ce0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:29.071686 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 889/1760] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:28 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:29.073084 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f68e959-a86e-4896-9d3e-40f2231a1e97 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:29.087316 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4f68e959-a86e-4896-9d3e-40f2231a1e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:29.151119 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:29.194430 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:29.250810 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 872/1761] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:29.253869 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923745fc-d7eb-4a95-b1e0-5277836ce0fb None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:29.261590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-923745fc-d7eb-4a95-b1e0-5277836ce0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:29.338444 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f68e959-a86e-4896-9d3e-40f2231a1e97 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:29.343594 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f286aff-c924-485a-b4bb-f8b96adf99d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:29.344312 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f286aff-c924-485a-b4bb-f8b96adf99d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:29.348255 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f286aff-c924-485a-b4bb-f8b96adf99d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:29.348255 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f286aff-c924-485a-b4bb-f8b96adf99d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:29.348255 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-9f286aff-c924-485a-b4bb-f8b96adf99d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:29.512345 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923745fc-d7eb-4a95-b1e0-5277836ce0fb None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:29.518792 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ccf7f2-64cd-4c4b-b1c9-6f04cee7f881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:29.519765 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccf7f2-64cd-4c4b-b1c9-6f04cee7f881 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:29.520103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccf7f2-64cd-4c4b-b1c9-6f04cee7f881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:29.520367 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccf7f2-64cd-4c4b-b1c9-6f04cee7f881 admin admin] PATH_INFO: `/v3/users/53bda2c82b4f470eab9043035e3a44b0` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:29.521889 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ccf7f2-64cd-4c4b-b1c9-6f04cee7f881 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53bda2c82b4f470eab9043035e3a44b0)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:29.524067 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ccf7f2-64cd-4c4b-b1c9-6f04cee7f881 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:29.557282 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f286aff-c924-485a-b4bb-f8b96adf99d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:29.561335 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f286aff-c924-485a-b4bb-f8b96adf99d6 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:29.614927 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 890/1762] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:29 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:29.616485 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f286aff-c924-485a-b4bb-f8b96adf99d6 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:29.618320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-9f286aff-c924-485a-b4bb-f8b96adf99d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:29.686025 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f286aff-c924-485a-b4bb-f8b96adf99d6 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:29.699410 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03210169-f9e1-49cc-b6da-d82d3f965bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:29.700334 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03210169-f9e1-49cc-b6da-d82d3f965bf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:29.700653 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03210169-f9e1-49cc-b6da-d82d3f965bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:29.701018 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03210169-f9e1-49cc-b6da-d82d3f965bf6 admin admin] PATH_INFO: `/v3/users/cd9203211bb144adb5bf25effea87dc7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:29.702579 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03210169-f9e1-49cc-b6da-d82d3f965bf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd9203211bb144adb5bf25effea87dc7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:29.704595 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03210169-f9e1-49cc-b6da-d82d3f965bf6 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:29.825801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b4ccf7f2-64cd-4c4b-b1c9-6f04cee7f881 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53bda2c82b4f470eab9043035e3a44b0'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:29.860713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b4ccf7f2-64cd-4c4b-b1c9-6f04cee7f881 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53bda2c82b4f470eab9043035e3a44b0'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:29.898404 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b4ccf7f2-64cd-4c4b-b1c9-6f04cee7f881 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53bda2c82b4f470eab9043035e3a44b0'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:29.899212 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b4ccf7f2-64cd-4c4b-b1c9-6f04cee7f881 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53bda2c82b4f470eab9043035e3a44b0'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:29.899845 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b4ccf7f2-64cd-4c4b-b1c9-6f04cee7f881 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53bda2c82b4f470eab9043035e3a44b0'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:29.960065 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-03210169-f9e1-49cc-b6da-d82d3f965bf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd9203211bb144adb5bf25effea87dc7'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:29.971317 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-03210169-f9e1-49cc-b6da-d82d3f965bf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd9203211bb144adb5bf25effea87dc7'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:29.971756 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-03210169-f9e1-49cc-b6da-d82d3f965bf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd9203211bb144adb5bf25effea87dc7'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:29.973815 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-03210169-f9e1-49cc-b6da-d82d3f965bf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd9203211bb144adb5bf25effea87dc7'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:30.013441 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 873/1763] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:26:29 2022] DELETE /identity/v3/users/53bda2c82b4f470eab9043035e3a44b0 => generated 0 bytes in 762 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:30.017366 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ccf7f2-64cd-4c4b-b1c9-6f04cee7f881 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:30.018203 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-03210169-f9e1-49cc-b6da-d82d3f965bf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd9203211bb144adb5bf25effea87dc7'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:30.029016 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b4ccf7f2-64cd-4c4b-b1c9-6f04cee7f881 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:30.167850 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 891/1764] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:26:29 2022] DELETE /identity/v3/users/cd9203211bb144adb5bf25effea87dc7 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:30.169415 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03210169-f9e1-49cc-b6da-d82d3f965bf6 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:30.180369 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-03210169-f9e1-49cc-b6da-d82d3f965bf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:30.256461 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ccf7f2-64cd-4c4b-b1c9-6f04cee7f881 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:30.404101 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:30.407495 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:30.456649 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17270fac-c37c-43fd-a823-c33983951a90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:30.459684 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17270fac-c37c-43fd-a823-c33983951a90 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:30.459684 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17270fac-c37c-43fd-a823-c33983951a90 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:30.459684 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17270fac-c37c-43fd-a823-c33983951a90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:30.467557 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-17270fac-c37c-43fd-a823-c33983951a90 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:30.478475 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03210169-f9e1-49cc-b6da-d82d3f965bf6 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:30.493931 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c09a18-f8a2-43a8-9b06-b37e48bfd5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:30.498959 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c09a18-f8a2-43a8-9b06-b37e48bfd5dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:30.498959 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c09a18-f8a2-43a8-9b06-b37e48bfd5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:30.499301 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c09a18-f8a2-43a8-9b06-b37e48bfd5dd admin admin] PATH_INFO: `/v3/projects/56f4eb50942b4e2eba5c3db4149a1e79` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:30.503252 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c09a18-f8a2-43a8-9b06-b37e48bfd5dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=56f4eb50942b4e2eba5c3db4149a1e79)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:30.508171 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c09a18-f8a2-43a8-9b06-b37e48bfd5dd admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:30.701597 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17270fac-c37c-43fd-a823-c33983951a90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:30.703966 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17270fac-c37c-43fd-a823-c33983951a90 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:30.726117 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 874/1765] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:30 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 712 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:30.735063 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e10af-2d75-463f-a845-840b4298f3f8 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:30.735717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e10af-2d75-463f-a845-840b4298f3f8 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:30.736005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e10af-2d75-463f-a845-840b4298f3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:30.749636 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-f0c09a18-f8a2-43a8-9b06-b37e48bfd5dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56f4eb50942b4e2eba5c3db4149a1e79', 'request_id': 'req-f0c09a18-f8a2-43a8-9b06-b37e48bfd5dd'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:30.767033 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-f0c09a18-f8a2-43a8-9b06-b37e48bfd5dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56f4eb50942b4e2eba5c3db4149a1e79', 'request_id': 'req-f0c09a18-f8a2-43a8-9b06-b37e48bfd5dd'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:30.836906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-e13e10af-2d75-463f-a845-840b4298f3f8 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:30.855733 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e13e10af-2d75-463f-a845-840b4298f3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:30.870675 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-f0c09a18-f8a2-43a8-9b06-b37e48bfd5dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56f4eb50942b4e2eba5c3db4149a1e79', 'request_id': 'req-f0c09a18-f8a2-43a8-9b06-b37e48bfd5dd'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:30.940603 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-f0c09a18-f8a2-43a8-9b06-b37e48bfd5dd admin admin] The token cache is being invalidate because project 56f4eb50942b4e2eba5c3db4149a1e79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:26:30.941134 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-f0c09a18-f8a2-43a8-9b06-b37e48bfd5dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:30.944502 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-f0c09a18-f8a2-43a8-9b06-b37e48bfd5dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:30.948215 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 892/1766] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:26:30 2022] DELETE /identity/v3/projects/56f4eb50942b4e2eba5c3db4149a1e79 => generated 0 bytes in 780 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:30.953631 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf6868-86ee-4880-8525-5916b0a2449b None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:30.954055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf6868-86ee-4880-8525-5916b0a2449b None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:30.956426 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf6868-86ee-4880-8525-5916b0a2449b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:30.960391 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 875/1767] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:30.986601 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-10cf6868-86ee-4880-8525-5916b0a2449b None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:30.986822 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13e10af-2d75-463f-a845-840b4298f3f8 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:30.988681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e13e10af-2d75-463f-a845-840b4298f3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:30.996672 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-10cf6868-86ee-4880-8525-5916b0a2449b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:31.021626 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 893/1768] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:26:31.023165 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2aa46-0030-4d3b-9e44-f8b08d58b1d3 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:31.023479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2aa46-0030-4d3b-9e44-f8b08d58b1d3 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:31.023752 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2aa46-0030-4d3b-9e44-f8b08d58b1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:31.063696 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-d1a2aa46-0030-4d3b-9e44-f8b08d58b1d3 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:31.065257 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d1a2aa46-0030-4d3b-9e44-f8b08d58b1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:31.078673 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 894/1769] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:31.079708 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a2aa46-0030-4d3b-9e44-f8b08d58b1d3 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:31.107260 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a2aa46-0030-4d3b-9e44-f8b08d58b1d3 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:31.116505 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf5db6e-10ff-482c-8073-67ef440f4f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:31.116505 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5db6e-10ff-482c-8073-67ef440f4f46 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:31.117382 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5db6e-10ff-482c-8073-67ef440f4f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:31.117382 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5db6e-10ff-482c-8073-67ef440f4f46 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:31.118755 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf5db6e-10ff-482c-8073-67ef440f4f46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:26:31.118755 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf5db6e-10ff-482c-8073-67ef440f4f46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:31.121599 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf5db6e-10ff-482c-8073-67ef440f4f46 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:31.159598 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13e10af-2d75-463f-a845-840b4298f3f8 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:31.171528 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28243b65-a27f-4172-a609-d7b2f3332e0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:31.172315 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28243b65-a27f-4172-a609-d7b2f3332e0b service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:31.172695 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28243b65-a27f-4172-a609-d7b2f3332e0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:31.172984 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28243b65-a27f-4172-a609-d7b2f3332e0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:31.184832 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 895/1770] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:26:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:26:31.188428 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf5db6e-10ff-482c-8073-67ef440f4f46 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:31.197983 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-7cf5db6e-10ff-482c-8073-67ef440f4f46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:31.200297 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28243b65-a27f-4172-a609-d7b2f3332e0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:31.202484 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28243b65-a27f-4172-a609-d7b2f3332e0b service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:31.217469 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 876/1771] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 06:26:31.218525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28243b65-a27f-4172-a609-d7b2f3332e0b service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:31.232868 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28243b65-a27f-4172-a609-d7b2f3332e0b service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:31.240899 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311ef97-6759-45a8-828b-239e5f6b595a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:31.241954 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311ef97-6759-45a8-828b-239e5f6b595a admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:31.242431 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311ef97-6759-45a8-828b-239e5f6b595a admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:31.242853 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311ef97-6759-45a8-828b-239e5f6b595a admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:31.243842 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6311ef97-6759-45a8-828b-239e5f6b595a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:31.246472 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6311ef97-6759-45a8-828b-239e5f6b595a admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:31.341119 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf5db6e-10ff-482c-8073-67ef440f4f46 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:31.353002 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb31eab-1c64-41a9-877a-4ccc6bdafa90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:31.354511 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb31eab-1c64-41a9-877a-4ccc6bdafa90 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:31.354511 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb31eab-1c64-41a9-877a-4ccc6bdafa90 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:31.354511 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb31eab-1c64-41a9-877a-4ccc6bdafa90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:31.361040 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-2cb31eab-1c64-41a9-877a-4ccc6bdafa90 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:31.426473 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 877/1772] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:26:31 2022] POST /identity/v3/projects => generated 403 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:31.432969 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311ef97-6759-45a8-828b-239e5f6b595a admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:31.493722 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:31.589089 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311ef97-6759-45a8-828b-239e5f6b595a admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:31.598939 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7f2e1-7bce-4c67-8819-bb6f82d6ee40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:31.599659 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7f2e1-7bce-4c67-8819-bb6f82d6ee40 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:31.599950 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7f2e1-7bce-4c67-8819-bb6f82d6ee40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:31.600244 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7f2e1-7bce-4c67-8819-bb6f82d6ee40 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:31.601897 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f7f2e1-7bce-4c67-8819-bb6f82d6ee40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:31.604255 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f7f2e1-7bce-4c67-8819-bb6f82d6ee40 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:31.625152 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb31eab-1c64-41a9-877a-4ccc6bdafa90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:31.628702 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb31eab-1c64-41a9-877a-4ccc6bdafa90 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:31.656839 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 896/1773] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:31.658466 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb31eab-1c64-41a9-877a-4ccc6bdafa90 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:31.660287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-2cb31eab-1c64-41a9-877a-4ccc6bdafa90 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:31.710104 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 878/1774] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:26:31 2022] POST /identity/v3/users => generated 302 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:26:31.713904 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7f2e1-7bce-4c67-8819-bb6f82d6ee40 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:31.717988 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb31eab-1c64-41a9-877a-4ccc6bdafa90 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:31.738253 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c62fca-f34e-4856-aec5-af8317541a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:31.739975 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7f2e1-7bce-4c67-8819-bb6f82d6ee40 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:31.741154 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c62fca-f34e-4856-aec5-af8317541a92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:31.741657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c62fca-f34e-4856-aec5-af8317541a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:31.742947 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c62fca-f34e-4856-aec5-af8317541a92 admin admin] PATH_INFO: `/v3/projects/6ec30bf602434097952a709fb5ca4665` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:31.746897 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c62fca-f34e-4856-aec5-af8317541a92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ec30bf602434097952a709fb5ca4665)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:31.753916 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c62fca-f34e-4856-aec5-af8317541a92 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:31.757790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae85d98-b9cd-4a62-b5fa-2142c631b3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:31.758906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae85d98-b9cd-4a62-b5fa-2142c631b3f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:31.759431 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae85d98-b9cd-4a62-b5fa-2142c631b3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:31.760652 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae85d98-b9cd-4a62-b5fa-2142c631b3f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:31.762330 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae85d98-b9cd-4a62-b5fa-2142c631b3f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:31.767202 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae85d98-b9cd-4a62-b5fa-2142c631b3f8 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:31.776358 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:31.813209 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 879/1775] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:31.821067 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae85d98-b9cd-4a62-b5fa-2142c631b3f8 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:31.856858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae85d98-b9cd-4a62-b5fa-2142c631b3f8 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:31.869483 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe0135-3433-4512-9187-0bb26c1ff84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:31.872732 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe0135-3433-4512-9187-0bb26c1ff84e admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:31.873496 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe0135-3433-4512-9187-0bb26c1ff84e admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:31.874225 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe0135-3433-4512-9187-0bb26c1ff84e admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:31.876222 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe0135-3433-4512-9187-0bb26c1ff84e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:31.890061 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe0135-3433-4512-9187-0bb26c1ff84e admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:31.921814 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 880/1776] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:31.924484 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe0135-3433-4512-9187-0bb26c1ff84e admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:31.945161 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe0135-3433-4512-9187-0bb26c1ff84e admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:31.954967 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d48f7-acb5-44da-952a-8f469ce57211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:31.957438 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d48f7-acb5-44da-952a-8f469ce57211 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:31.958127 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d48f7-acb5-44da-952a-8f469ce57211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:31.958639 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d48f7-acb5-44da-952a-8f469ce57211 admin admin] PATH_INFO: `/v3/projects/0db668553be14231aea54d7d51c0def6/users/ffc85a80354d4d1c8cc066c519ea29a7/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:32.033746 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-81c62fca-f34e-4856-aec5-af8317541a92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ec30bf602434097952a709fb5ca4665', 'request_id': 'req-81c62fca-f34e-4856-aec5-af8317541a92'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:32.057979 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-81c62fca-f34e-4856-aec5-af8317541a92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ec30bf602434097952a709fb5ca4665', 'request_id': 'req-81c62fca-f34e-4856-aec5-af8317541a92'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:32.081183 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485d48f7-acb5-44da-952a-8f469ce57211 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0db668553be14231aea54d7d51c0def6, user_id=ffc85a80354d4d1c8cc066c519ea29a7, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:32.121333 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485d48f7-acb5-44da-952a-8f469ce57211 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:32.165901 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-81c62fca-f34e-4856-aec5-af8317541a92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ec30bf602434097952a709fb5ca4665', 'request_id': 'req-81c62fca-f34e-4856-aec5-af8317541a92'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:32.183834 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-81c62fca-f34e-4856-aec5-af8317541a92 admin admin] The token cache is being invalidate because project 6ec30bf602434097952a709fb5ca4665 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:26:32.184271 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-81c62fca-f34e-4856-aec5-af8317541a92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:32.186892 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-81c62fca-f34e-4856-aec5-af8317541a92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:32.190578 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 897/1777] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:26:31 2022] DELETE /identity/v3/projects/6ec30bf602434097952a709fb5ca4665 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:32.201454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd25cb-c695-4263-96bd-f32518dd3080 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:32.202165 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd25cb-c695-4263-96bd-f32518dd3080 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:32.202710 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd25cb-c695-4263-96bd-f32518dd3080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:32.234915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-b2cd25cb-c695-4263-96bd-f32518dd3080 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:32.236578 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-b2cd25cb-c695-4263-96bd-f32518dd3080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:32.252737 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 881/1778] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:31 2022] PUT /identity/v3/projects/0db668553be14231aea54d7d51c0def6/users/ffc85a80354d4d1c8cc066c519ea29a7/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:32.260581 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd3e9e-beb2-4506-9b55-f4ca1a1c9bd7 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:32.260886 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd3e9e-beb2-4506-9b55-f4ca1a1c9bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:32.261238 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd3e9e-beb2-4506-9b55-f4ca1a1c9bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:32.287311 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-66bd3e9e-beb2-4506-9b55-f4ca1a1c9bd7 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:32.289846 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-66bd3e9e-beb2-4506-9b55-f4ca1a1c9bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:32.456076 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 898/1779] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:32.464791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bc2f5-f533-4571-9326-905cc40a5684 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:32.465128 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bc2f5-f533-4571-9326-905cc40a5684 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:32.465465 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bc2f5-f533-4571-9326-905cc40a5684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:32.474514 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 882/1780] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:32.485511 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bd3e9e-beb2-4506-9b55-f4ca1a1c9bd7 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:32.487241 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-66bd3e9e-beb2-4506-9b55-f4ca1a1c9bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:32.511066 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-fc4bc2f5-f533-4571-9326-905cc40a5684 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:32.513033 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-fc4bc2f5-f533-4571-9326-905cc40a5684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:32.533474 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 899/1781] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:32.542081 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4bc2f5-f533-4571-9326-905cc40a5684 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:32.567422 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4bc2f5-f533-4571-9326-905cc40a5684 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:32.582331 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19059dc3-0a93-444d-81ed-885c6d772ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:32.583705 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19059dc3-0a93-444d-81ed-885c6d772ec7 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:32.584106 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19059dc3-0a93-444d-81ed-885c6d772ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:32.584578 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19059dc3-0a93-444d-81ed-885c6d772ec7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:32.585456 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19059dc3-0a93-444d-81ed-885c6d772ec7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:26:32.586020 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19059dc3-0a93-444d-81ed-885c6d772ec7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:32.599184 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19059dc3-0a93-444d-81ed-885c6d772ec7 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:32.630982 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:32.632972 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bd3e9e-beb2-4506-9b55-f4ca1a1c9bd7 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:32.638824 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8325013b-29c3-4180-a824-cdca0ffba9cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:32.639512 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8325013b-29c3-4180-a824-cdca0ffba9cd service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:32.639802 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8325013b-29c3-4180-a824-cdca0ffba9cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:32.640959 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8325013b-29c3-4180-a824-cdca0ffba9cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:32.670856 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 900/1782] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:26:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:26:32.678704 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19059dc3-0a93-444d-81ed-885c6d772ec7 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:32.681404 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8325013b-29c3-4180-a824-cdca0ffba9cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:32.683380 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8325013b-29c3-4180-a824-cdca0ffba9cd service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:32.691482 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-19059dc3-0a93-444d-81ed-885c6d772ec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:32.720514 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 883/1783] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:32.724099 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8325013b-29c3-4180-a824-cdca0ffba9cd service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:32.734484 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19059dc3-0a93-444d-81ed-885c6d772ec7 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:32.753989 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67efd666-4cfd-4ba0-807c-2d354529f5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:32.755618 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67efd666-4cfd-4ba0-807c-2d354529f5db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:32.757283 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67efd666-4cfd-4ba0-807c-2d354529f5db admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:32.757751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67efd666-4cfd-4ba0-807c-2d354529f5db admin admin] PATH_INFO: `/v3/projects/19737acd3c0542bd9d0c8978b1d98657` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:32.759629 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8325013b-29c3-4180-a824-cdca0ffba9cd service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:32.760558 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67efd666-4cfd-4ba0-807c-2d354529f5db admin admin] RBAC: Authorizing `identity:delete_project(project_id=19737acd3c0542bd9d0c8978b1d98657)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:32.763100 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67efd666-4cfd-4ba0-807c-2d354529f5db admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:32.772696 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d189382a-f129-44fd-827d-3cc9ac4f2444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:32.775004 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d189382a-f129-44fd-827d-3cc9ac4f2444 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:32.776525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d189382a-f129-44fd-827d-3cc9ac4f2444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:32.778723 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d189382a-f129-44fd-827d-3cc9ac4f2444 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:32.780759 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d189382a-f129-44fd-827d-3cc9ac4f2444 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:32.788461 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d189382a-f129-44fd-827d-3cc9ac4f2444 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:32.977835 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-67efd666-4cfd-4ba0-807c-2d354529f5db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19737acd3c0542bd9d0c8978b1d98657', 'request_id': 'req-67efd666-4cfd-4ba0-807c-2d354529f5db'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:32.999988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 884/1784] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:26:32 2022] POST /identity/v3/projects => generated 381 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:33.006569 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d189382a-f129-44fd-827d-3cc9ac4f2444 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:33.010977 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-67efd666-4cfd-4ba0-807c-2d354529f5db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19737acd3c0542bd9d0c8978b1d98657', 'request_id': 'req-67efd666-4cfd-4ba0-807c-2d354529f5db'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:33.126020 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-67efd666-4cfd-4ba0-807c-2d354529f5db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19737acd3c0542bd9d0c8978b1d98657', 'request_id': 'req-67efd666-4cfd-4ba0-807c-2d354529f5db'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:33.140696 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d189382a-f129-44fd-827d-3cc9ac4f2444 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:33.146357 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-67efd666-4cfd-4ba0-807c-2d354529f5db admin admin] The token cache is being invalidate because project 19737acd3c0542bd9d0c8978b1d98657 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:26:33.147023 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-67efd666-4cfd-4ba0-807c-2d354529f5db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:33.148249 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-67efd666-4cfd-4ba0-807c-2d354529f5db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:33.149766 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ea991-17df-4bee-b548-081beac7dd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:33.150776 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ea991-17df-4bee-b548-081beac7dd6f admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:33.151204 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ea991-17df-4bee-b548-081beac7dd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:33.151517 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ea991-17df-4bee-b548-081beac7dd6f admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:33.152778 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 901/1785] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:26:32 2022] DELETE /identity/v3/projects/19737acd3c0542bd9d0c8978b1d98657 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:33.152919 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9ea991-17df-4bee-b548-081beac7dd6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:33.154828 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9ea991-17df-4bee-b548-081beac7dd6f admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:33.157202 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67efd666-4cfd-4ba0-807c-2d354529f5db admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:33.159352 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-67efd666-4cfd-4ba0-807c-2d354529f5db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:33.294586 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67efd666-4cfd-4ba0-807c-2d354529f5db admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:33.302753 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 885/1786] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:26:33 2022] POST /identity/v3/users => generated 291 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:26:33.308606 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ea991-17df-4bee-b548-081beac7dd6f admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:33.311401 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-cb9ea991-17df-4bee-b548-081beac7dd6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:33.312027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b75de3-8789-40c2-ab84-a30c95f6d533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:33.312958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b75de3-8789-40c2-ab84-a30c95f6d533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:33.313254 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b75de3-8789-40c2-ab84-a30c95f6d533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:33.313570 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b75de3-8789-40c2-ab84-a30c95f6d533 admin admin] PATH_INFO: `/v3/users/a378fd60c6af4778a443a21f5bcca46a` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:33.319197 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b75de3-8789-40c2-ab84-a30c95f6d533 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a378fd60c6af4778a443a21f5bcca46a)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:33.321461 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b75de3-8789-40c2-ab84-a30c95f6d533 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:33.343825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ea991-17df-4bee-b548-081beac7dd6f admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:33.360548 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:33.366128 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcd2578-4fd0-460a-827e-ea953644aad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:33.376930 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd2578-4fd0-460a-827e-ea953644aad8 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:33.377560 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd2578-4fd0-460a-827e-ea953644aad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:33.378503 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd2578-4fd0-460a-827e-ea953644aad8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:33.382087 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcd2578-4fd0-460a-827e-ea953644aad8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:33.396726 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcd2578-4fd0-460a-827e-ea953644aad8 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:33.417094 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 886/1787] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:33.421650 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcd2578-4fd0-460a-827e-ea953644aad8 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:33.434927 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcd2578-4fd0-460a-827e-ea953644aad8 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:33.445672 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa8279-bf3e-4135-bb26-8264ec1e3eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:33.446848 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa8279-bf3e-4135-bb26-8264ec1e3eb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:33.447348 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa8279-bf3e-4135-bb26-8264ec1e3eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:33.447834 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa8279-bf3e-4135-bb26-8264ec1e3eb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:33.449303 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa8279-bf3e-4135-bb26-8264ec1e3eb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:33.452480 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa8279-bf3e-4135-bb26-8264ec1e3eb1 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:33.476331 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 887/1788] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 06:26:33.480638 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa8279-bf3e-4135-bb26-8264ec1e3eb1 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:33.492392 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa8279-bf3e-4135-bb26-8264ec1e3eb1 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:33.498207 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14b7d0-0439-4863-b4a1-8aebf21deec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:33.499228 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14b7d0-0439-4863-b4a1-8aebf21deec1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:33.499500 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14b7d0-0439-4863-b4a1-8aebf21deec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:33.499789 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14b7d0-0439-4863-b4a1-8aebf21deec1 admin admin] PATH_INFO: `/v3/projects/83993250dab64203bcefa94e341f6630/users/a92c16fc4ba049ec8aa08c2ef9af31c9/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:33.525332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d14b7d0-0439-4863-b4a1-8aebf21deec1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83993250dab64203bcefa94e341f6630, user_id=a92c16fc4ba049ec8aa08c2ef9af31c9, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:33.530805 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d14b7d0-0439-4863-b4a1-8aebf21deec1 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:33.614995 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e2b75de3-8789-40c2-ab84-a30c95f6d533 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a378fd60c6af4778a443a21f5bcca46a'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:33.626652 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e2b75de3-8789-40c2-ab84-a30c95f6d533 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a378fd60c6af4778a443a21f5bcca46a'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:33.628027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e2b75de3-8789-40c2-ab84-a30c95f6d533 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a378fd60c6af4778a443a21f5bcca46a'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:33.628829 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e2b75de3-8789-40c2-ab84-a30c95f6d533 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a378fd60c6af4778a443a21f5bcca46a'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:33.673526 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e2b75de3-8789-40c2-ab84-a30c95f6d533 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a378fd60c6af4778a443a21f5bcca46a'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:33.717002 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 888/1789] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:33 2022] PUT /identity/v3/projects/83993250dab64203bcefa94e341f6630/users/a92c16fc4ba049ec8aa08c2ef9af31c9/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:26:33.736910 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720884e7-2da9-412e-a5d4-33a71ac4f0ea None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:33.737394 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720884e7-2da9-412e-a5d4-33a71ac4f0ea None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:33.737531 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720884e7-2da9-412e-a5d4-33a71ac4f0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:33.792510 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-720884e7-2da9-412e-a5d4-33a71ac4f0ea None None] MFA Rules not processed for user `a92c16fc4ba049ec8aa08c2ef9af31c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:33.793835 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-720884e7-2da9-412e-a5d4-33a71ac4f0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:33.835389 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 902/1790] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:26:33 2022] DELETE /identity/v3/users/a378fd60c6af4778a443a21f5bcca46a => generated 0 bytes in 679 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:26:33.950704 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:33.965912 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 889/1791] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:33.968914 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b75de3-8789-40c2-ab84-a30c95f6d533 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:33.970667 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e2b75de3-8789-40c2-ab84-a30c95f6d533 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:34.077836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b75de3-8789-40c2-ab84-a30c95f6d533 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:34.108108 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7904f1-f6aa-4d70-8fc2-76a8314539dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:34.108929 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7904f1-f6aa-4d70-8fc2-76a8314539dd admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:34.112604 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7904f1-f6aa-4d70-8fc2-76a8314539dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:34.118255 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7904f1-f6aa-4d70-8fc2-76a8314539dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:34.118255 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7904f1-f6aa-4d70-8fc2-76a8314539dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:34.122623 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7904f1-f6aa-4d70-8fc2-76a8314539dd admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:34.312762 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 903/1792] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:26:33 2022] POST /identity/v3/projects => generated 383 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:34.319009 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720884e7-2da9-412e-a5d4-33a71ac4f0ea None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:34.472586 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720884e7-2da9-412e-a5d4-33a71ac4f0ea None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:34.593149 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b80e4b-4843-4877-81a0-b9d6cf3d2f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:34.595452 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b80e4b-4843-4877-81a0-b9d6cf3d2f04 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:34.596452 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b80e4b-4843-4877-81a0-b9d6cf3d2f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:34.598643 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b80e4b-4843-4877-81a0-b9d6cf3d2f04 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:34.600388 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b80e4b-4843-4877-81a0-b9d6cf3d2f04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:34.628864 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b80e4b-4843-4877-81a0-b9d6cf3d2f04 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:34.787430 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 890/1793] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:26:34 2022] POST /identity/v3/users => generated 298 bytes in 467 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:34.789466 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7904f1-f6aa-4d70-8fc2-76a8314539dd admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:34.818439 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7904f1-f6aa-4d70-8fc2-76a8314539dd admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:34.833713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b80e4b-4843-4877-81a0-b9d6cf3d2f04 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:34.835994 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a0b80e4b-4843-4877-81a0-b9d6cf3d2f04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:34.843236 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f101fb8-f6a0-40e0-9d31-109c6c9cb2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:34.844465 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f101fb8-f6a0-40e0-9d31-109c6c9cb2d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:34.846571 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f101fb8-f6a0-40e0-9d31-109c6c9cb2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:34.847417 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f101fb8-f6a0-40e0-9d31-109c6c9cb2d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:34.849913 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f101fb8-f6a0-40e0-9d31-109c6c9cb2d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:34.854424 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f101fb8-f6a0-40e0-9d31-109c6c9cb2d9 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:34.865362 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b80e4b-4843-4877-81a0-b9d6cf3d2f04 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:34.874113 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b4cf4c-7f11-41c0-98d9-427a37966d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:34.875520 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b4cf4c-7f11-41c0-98d9-427a37966d50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:34.876437 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b4cf4c-7f11-41c0-98d9-427a37966d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:34.876813 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b4cf4c-7f11-41c0-98d9-427a37966d50 admin admin] PATH_INFO: `/v3/users/f3ce2bd4b7da434faa9bbd4563042d25` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:34.878438 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b4cf4c-7f11-41c0-98d9-427a37966d50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3ce2bd4b7da434faa9bbd4563042d25)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:34.881056 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b4cf4c-7f11-41c0-98d9-427a37966d50 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:34.883275 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 904/1794] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:34 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:34.887938 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f101fb8-f6a0-40e0-9d31-109c6c9cb2d9 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:34.915955 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f101fb8-f6a0-40e0-9d31-109c6c9cb2d9 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:34.947064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c4f95-aaee-4ea0-be52-5cb1d7850ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:34.954553 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c4f95-aaee-4ea0-be52-5cb1d7850ffb admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:34.955613 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c4f95-aaee-4ea0-be52-5cb1d7850ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:34.956932 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c4f95-aaee-4ea0-be52-5cb1d7850ffb admin admin] PATH_INFO: `/v3/projects/cb2fdea479d44caea280e8a747e12ddc/users/454eacbc2c4341e79ab33f6926d2d035/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:35.001982 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996c4f95-aaee-4ea0-be52-5cb1d7850ffb admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb2fdea479d44caea280e8a747e12ddc, user_id=454eacbc2c4341e79ab33f6926d2d035, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:35.009013 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996c4f95-aaee-4ea0-be52-5cb1d7850ffb admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:35.142543 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 905/1795] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:34 2022] PUT /identity/v3/projects/cb2fdea479d44caea280e8a747e12ddc/users/454eacbc2c4341e79ab33f6926d2d035/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:26:35.146114 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218a052-9d1f-4f93-911a-17857b32de57 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:35.147667 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218a052-9d1f-4f93-911a-17857b32de57 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:35.148063 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218a052-9d1f-4f93-911a-17857b32de57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:35.211863 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-64b4cf4c-7f11-41c0-98d9-427a37966d50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3ce2bd4b7da434faa9bbd4563042d25'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:35.231296 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-64b4cf4c-7f11-41c0-98d9-427a37966d50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3ce2bd4b7da434faa9bbd4563042d25'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:35.270954 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:35.275567 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-64b4cf4c-7f11-41c0-98d9-427a37966d50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3ce2bd4b7da434faa9bbd4563042d25'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:35.278697 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-64b4cf4c-7f11-41c0-98d9-427a37966d50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3ce2bd4b7da434faa9bbd4563042d25'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:35.278697 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-64b4cf4c-7f11-41c0-98d9-427a37966d50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3ce2bd4b7da434faa9bbd4563042d25'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:35.329888 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-b218a052-9d1f-4f93-911a-17857b32de57 None None] MFA Rules not processed for user `b84dde57ff8e47419866377a819bea21`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:35.335992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-b218a052-9d1f-4f93-911a-17857b32de57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:35.395679 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 891/1796] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:26:34 2022] DELETE /identity/v3/users/f3ce2bd4b7da434faa9bbd4563042d25 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:26:35.396150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b4cf4c-7f11-41c0-98d9-427a37966d50 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:35.404098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-64b4cf4c-7f11-41c0-98d9-427a37966d50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:35.485113 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:35.523510 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 906/1797] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Jul 18 06:26:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:35.525372 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038c607-b322-4a78-8818-5a994f93602a None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:35.525945 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038c607-b322-4a78-8818-5a994f93602a None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:35.526327 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038c607-b322-4a78-8818-5a994f93602a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:35.554124 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-7038c607-b322-4a78-8818-5a994f93602a None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:35.556245 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-7038c607-b322-4a78-8818-5a994f93602a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:35.612178 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b4cf4c-7f11-41c0-98d9-427a37966d50 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:35.630367 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d6c0cf-5876-4d93-b516-21fcb31c8450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:35.631079 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6c0cf-5876-4d93-b516-21fcb31c8450 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:35.631369 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6c0cf-5876-4d93-b516-21fcb31c8450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:35.631650 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6c0cf-5876-4d93-b516-21fcb31c8450 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:35.632850 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d6c0cf-5876-4d93-b516-21fcb31c8450 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:35.634805 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d6c0cf-5876-4d93-b516-21fcb31c8450 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:35.673382 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 892/1798] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:35.675000 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d6c0cf-5876-4d93-b516-21fcb31c8450 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:35.698057 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 907/1799] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:35.709712 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d6c0cf-5876-4d93-b516-21fcb31c8450 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:35.728247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333bf238-1e4a-49d0-94f1-2f456b047069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:35.728247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333bf238-1e4a-49d0-94f1-2f456b047069 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:35.728247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333bf238-1e4a-49d0-94f1-2f456b047069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:35.729134 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333bf238-1e4a-49d0-94f1-2f456b047069 admin admin] PATH_INFO: `/v3/projects/cb2fdea479d44caea280e8a747e12ddc/users/454eacbc2c4341e79ab33f6926d2d035/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:35.734415 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333bf238-1e4a-49d0-94f1-2f456b047069 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb2fdea479d44caea280e8a747e12ddc, user_id=454eacbc2c4341e79ab33f6926d2d035, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:35.737243 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333bf238-1e4a-49d0-94f1-2f456b047069 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:35.742853 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038c607-b322-4a78-8818-5a994f93602a None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:35.745402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-7038c607-b322-4a78-8818-5a994f93602a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:35.756308 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 893/1800] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:35 2022] PUT /identity/v3/projects/cb2fdea479d44caea280e8a747e12ddc/users/454eacbc2c4341e79ab33f6926d2d035/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:35.766494 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333bf238-1e4a-49d0-94f1-2f456b047069 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:35.768339 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-333bf238-1e4a-49d0-94f1-2f456b047069 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:35.782679 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038c607-b322-4a78-8818-5a994f93602a None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:35.976862 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333bf238-1e4a-49d0-94f1-2f456b047069 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:35.985169 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a274f-5a95-43aa-b46a-bc8a42024ce3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:35.986700 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a274f-5a95-43aa-b46a-bc8a42024ce3 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:35.987297 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a274f-5a95-43aa-b46a-bc8a42024ce3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:35.987817 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a274f-5a95-43aa-b46a-bc8a42024ce3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:36.009490 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4adf443-d7e3-4320-9bd1-d5c50b579e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:36.014066 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4adf443-d7e3-4320-9bd1-d5c50b579e6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:36.030231 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4adf443-d7e3-4320-9bd1-d5c50b579e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:36.030646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4adf443-d7e3-4320-9bd1-d5c50b579e6c admin admin] PATH_INFO: `/v3/projects/adaf93a38977429db9cfae6ee619e0a8` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:36.037103 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4adf443-d7e3-4320-9bd1-d5c50b579e6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=adaf93a38977429db9cfae6ee619e0a8)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:36.042385 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4adf443-d7e3-4320-9bd1-d5c50b579e6c admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:36.154835 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a274f-5a95-43aa-b46a-bc8a42024ce3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:36.166920 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a274f-5a95-43aa-b46a-bc8a42024ce3 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:36.193011 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 894/1801] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:36.201104 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0be40e-3297-4399-a9b2-e56065c1860b None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:36.201406 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0be40e-3297-4399-a9b2-e56065c1860b None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:36.201695 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0be40e-3297-4399-a9b2-e56065c1860b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:36.238077 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-6a0be40e-3297-4399-a9b2-e56065c1860b None None] MFA Rules not processed for user `454eacbc2c4341e79ab33f6926d2d035`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:36.239884 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-6a0be40e-3297-4399-a9b2-e56065c1860b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:36.249135 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-b4adf443-d7e3-4320-9bd1-d5c50b579e6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adaf93a38977429db9cfae6ee619e0a8', 'request_id': 'req-b4adf443-d7e3-4320-9bd1-d5c50b579e6c'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:36.263888 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-b4adf443-d7e3-4320-9bd1-d5c50b579e6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adaf93a38977429db9cfae6ee619e0a8', 'request_id': 'req-b4adf443-d7e3-4320-9bd1-d5c50b579e6c'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:36.372817 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 895/1802] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:36 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:36.374821 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0be40e-3297-4399-a9b2-e56065c1860b None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:36.389211 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-6a0be40e-3297-4399-a9b2-e56065c1860b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:36.399652 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-b4adf443-d7e3-4320-9bd1-d5c50b579e6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adaf93a38977429db9cfae6ee619e0a8', 'request_id': 'req-b4adf443-d7e3-4320-9bd1-d5c50b579e6c'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:36.461233 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-b4adf443-d7e3-4320-9bd1-d5c50b579e6c admin admin] The token cache is being invalidate because project adaf93a38977429db9cfae6ee619e0a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:26:36.463913 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-b4adf443-d7e3-4320-9bd1-d5c50b579e6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:36.470514 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-b4adf443-d7e3-4320-9bd1-d5c50b579e6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:36.474573 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 908/1803] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:26:35 2022] DELETE /identity/v3/projects/adaf93a38977429db9cfae6ee619e0a8 => generated 0 bytes in 732 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:36.476531 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f522839-37d9-41cd-b733-7472d2fbeaa8 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:36.477096 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f522839-37d9-41cd-b733-7472d2fbeaa8 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:36.477641 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f522839-37d9-41cd-b733-7472d2fbeaa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:36.524999 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-4f522839-37d9-41cd-b733-7472d2fbeaa8 None None] MFA Rules not processed for user `ffc85a80354d4d1c8cc066c519ea29a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:36.526447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4f522839-37d9-41cd-b733-7472d2fbeaa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:36.618763 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0be40e-3297-4399-a9b2-e56065c1860b None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:36.634104 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c9fe2-c3be-41eb-a3e6-4b6ffb6b7dc4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0681f5fbea6d415c9ad2f3afae256a5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:36.635158 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c9fe2-c3be-41eb-a3e6-4b6ffb6b7dc4 service swift] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:36.636258 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c9fe2-c3be-41eb-a3e6-4b6ffb6b7dc4 service swift] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:36.636781 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c9fe2-c3be-41eb-a3e6-4b6ffb6b7dc4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:36.639844 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f40c9fe2-c3be-41eb-a3e6-4b6ffb6b7dc4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:36.687811 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 909/1804] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:36 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:36.689428 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f522839-37d9-41cd-b733-7472d2fbeaa8 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:36.696088 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4f522839-37d9-41cd-b733-7472d2fbeaa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:36.830564 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f522839-37d9-41cd-b733-7472d2fbeaa8 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:36.836753 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bcad67-080e-49da-a1f8-ed29ab7b4ab7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:36.837448 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bcad67-080e-49da-a1f8-ed29ab7b4ab7 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:36.838454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bcad67-080e-49da-a1f8-ed29ab7b4ab7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:36.839010 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bcad67-080e-49da-a1f8-ed29ab7b4ab7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:36.841142 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-17bcad67-080e-49da-a1f8-ed29ab7b4ab7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:36.849735 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40c9fe2-c3be-41eb-a3e6-4b6ffb6b7dc4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:36.852838 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40c9fe2-c3be-41eb-a3e6-4b6ffb6b7dc4 service swift] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:36.856131 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bcad67-080e-49da-a1f8-ed29ab7b4ab7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:36.857738 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bcad67-080e-49da-a1f8-ed29ab7b4ab7 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:36.870484 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 896/1805] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Jul 18 06:26:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 06:26:36.872232 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558756a-8932-4322-89b9-61668cbd9e4a None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:36.872955 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558756a-8932-4322-89b9-61668cbd9e4a None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:36.872955 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558756a-8932-4322-89b9-61668cbd9e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:36.874958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 910/1806] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:36 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:36.876158 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83ad14-8daf-4624-8ab2-7626e88377c9 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:36.876849 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83ad14-8daf-4624-8ab2-7626e88377c9 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:36.877183 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83ad14-8daf-4624-8ab2-7626e88377c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:36.905619 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-aa83ad14-8daf-4624-8ab2-7626e88377c9 None None] MFA Rules not processed for user `9824a96e2bcd4a16bfc6fe6bf4ea1ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:36.905619 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-aa83ad14-8daf-4624-8ab2-7626e88377c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:36.923232 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-8558756a-8932-4322-89b9-61668cbd9e4a None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:36.924497 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8558756a-8932-4322-89b9-61668cbd9e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:37.080622 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 911/1807] 10.222.0.17 () {58 vars in 1015 bytes} [Mon Jul 18 06:26:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:37.081916 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa83ad14-8daf-4624-8ab2-7626e88377c9 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:37.087173 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-aa83ad14-8daf-4624-8ab2-7626e88377c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:37.136752 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 897/1808] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:37.138093 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558756a-8932-4322-89b9-61668cbd9e4a None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:37.161912 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558756a-8932-4322-89b9-61668cbd9e4a None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:37.169591 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986e2328-b871-42e4-93f1-995d64148901 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:37.175428 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e2328-b871-42e4-93f1-995d64148901 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:37.175428 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e2328-b871-42e4-93f1-995d64148901 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:37.175428 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e2328-b871-42e4-93f1-995d64148901 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:37.175428 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-986e2328-b871-42e4-93f1-995d64148901 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:37.233749 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:37.410030 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa83ad14-8daf-4624-8ab2-7626e88377c9 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:37.427333 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a213c2d-0b78-4e67-95bc-f1bebd5944f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:37.429163 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a213c2d-0b78-4e67-95bc-f1bebd5944f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:37.433939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a213c2d-0b78-4e67-95bc-f1bebd5944f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:37.433939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a213c2d-0b78-4e67-95bc-f1bebd5944f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:37.433939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a213c2d-0b78-4e67-95bc-f1bebd5944f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:37.438995 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a213c2d-0b78-4e67-95bc-f1bebd5944f9 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:37.454158 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986e2328-b871-42e4-93f1-995d64148901 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:37.455713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986e2328-b871-42e4-93f1-995d64148901 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:37.502401 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 898/1809] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:37.503481 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986e2328-b871-42e4-93f1-995d64148901 service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:37.509394 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-986e2328-b871-42e4-93f1-995d64148901 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:37.562128 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986e2328-b871-42e4-93f1-995d64148901 service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:37.574822 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb69843f-060c-43e9-bb6e-a67037007b06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:37.575526 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb69843f-060c-43e9-bb6e-a67037007b06 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:37.577310 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb69843f-060c-43e9-bb6e-a67037007b06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:37.577650 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb69843f-060c-43e9-bb6e-a67037007b06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:37.586309 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:37.592870 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 912/1810] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:26:37 2022] POST /identity/v3/projects => generated 405 bytes in 511 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:37.594921 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb69843f-060c-43e9-bb6e-a67037007b06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:37.595578 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec1c5a-ac10-40be-bdbc-87a639e53c85 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:37.596399 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec1c5a-ac10-40be-bdbc-87a639e53c85 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:37.596659 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb69843f-060c-43e9-bb6e-a67037007b06 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:37.597544 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec1c5a-ac10-40be-bdbc-87a639e53c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:37.608388 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 899/1811] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:37.609314 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb69843f-060c-43e9-bb6e-a67037007b06 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:37.624120 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb69843f-060c-43e9-bb6e-a67037007b06 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:37.659920 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-74ec1c5a-ac10-40be-bdbc-87a639e53c85 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:37.671909 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-74ec1c5a-ac10-40be-bdbc-87a639e53c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:37.708744 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67923df-947c-489e-85c8-468008145127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:37.709567 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67923df-947c-489e-85c8-468008145127 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:37.710008 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67923df-947c-489e-85c8-468008145127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:37.710684 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67923df-947c-489e-85c8-468008145127 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:37.713686 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67923df-947c-489e-85c8-468008145127 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:37.713686 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67923df-947c-489e-85c8-468008145127 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:37.835422 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 900/1812] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:26:37 2022] POST /identity/v3/users => generated 309 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:37.835422 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67923df-947c-489e-85c8-468008145127 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:37.848851 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e67923df-947c-489e-85c8-468008145127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:37.875271 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 913/1813] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:37.877170 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec1c5a-ac10-40be-bdbc-87a639e53c85 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:37.881996 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67923df-947c-489e-85c8-468008145127 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:37.903796 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec1c5a-ac10-40be-bdbc-87a639e53c85 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:37.905291 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d478c19-8d0f-4079-ad05-1f8ea0f145e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:37.906204 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d478c19-8d0f-4079-ad05-1f8ea0f145e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:37.906499 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d478c19-8d0f-4079-ad05-1f8ea0f145e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:37.912959 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d478c19-8d0f-4079-ad05-1f8ea0f145e0 admin admin] PATH_INFO: `/v3/projects/3f3b2caf25394630b17da048591d2fe8` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:37.919483 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d478c19-8d0f-4079-ad05-1f8ea0f145e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f3b2caf25394630b17da048591d2fe8)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:37.920356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a9e71-4a9e-4618-ab82-82e3f20d9c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:37.921407 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a9e71-4a9e-4618-ab82-82e3f20d9c91 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:37.922278 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a9e71-4a9e-4618-ab82-82e3f20d9c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:37.922759 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a9e71-4a9e-4618-ab82-82e3f20d9c91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:37.924391 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d478c19-8d0f-4079-ad05-1f8ea0f145e0 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:37.925219 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a9e71-4a9e-4618-ab82-82e3f20d9c91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:37.930656 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a9e71-4a9e-4618-ab82-82e3f20d9c91 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:37.961875 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 914/1814] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:37 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:37.964629 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a9e71-4a9e-4618-ab82-82e3f20d9c91 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:38.008101 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a9e71-4a9e-4618-ab82-82e3f20d9c91 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:38.022436 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c16a7f-3cf1-4f06-80fa-df0399ba1c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:38.023172 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c16a7f-3cf1-4f06-80fa-df0399ba1c51 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:38.023465 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c16a7f-3cf1-4f06-80fa-df0399ba1c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:38.023750 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c16a7f-3cf1-4f06-80fa-df0399ba1c51 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:38.024569 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c16a7f-3cf1-4f06-80fa-df0399ba1c51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:26:38.024891 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c16a7f-3cf1-4f06-80fa-df0399ba1c51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:38.030835 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c16a7f-3cf1-4f06-80fa-df0399ba1c51 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:38.071401 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 915/1815] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:26:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:26:38.073017 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c16a7f-3cf1-4f06-80fa-df0399ba1c51 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:38.109082 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c16a7f-3cf1-4f06-80fa-df0399ba1c51 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:38.126770 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25abd28d-e040-48f9-a635-5582034e0591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:38.128783 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25abd28d-e040-48f9-a635-5582034e0591 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:38.129659 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25abd28d-e040-48f9-a635-5582034e0591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:38.130870 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25abd28d-e040-48f9-a635-5582034e0591 admin admin] PATH_INFO: `/v3/projects/412e6396ca0641379f1f09e5ce1fb2bb/users/7bcf963d78854e8a892e61e049f254bd/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:38.170427 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9d478c19-8d0f-4079-ad05-1f8ea0f145e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f3b2caf25394630b17da048591d2fe8', 'request_id': 'req-9d478c19-8d0f-4079-ad05-1f8ea0f145e0'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:38.180888 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9d478c19-8d0f-4079-ad05-1f8ea0f145e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f3b2caf25394630b17da048591d2fe8', 'request_id': 'req-9d478c19-8d0f-4079-ad05-1f8ea0f145e0'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:38.194121 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25abd28d-e040-48f9-a635-5582034e0591 admin admin] RBAC: Authorizing `identity:create_grant(project_id=412e6396ca0641379f1f09e5ce1fb2bb, user_id=7bcf963d78854e8a892e61e049f254bd, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:38.202160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25abd28d-e040-48f9-a635-5582034e0591 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:38.295652 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:38.329996 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9d478c19-8d0f-4079-ad05-1f8ea0f145e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f3b2caf25394630b17da048591d2fe8', 'request_id': 'req-9d478c19-8d0f-4079-ad05-1f8ea0f145e0'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:38.360848 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 916/1816] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:38 2022] PUT /identity/v3/projects/412e6396ca0641379f1f09e5ce1fb2bb/users/7bcf963d78854e8a892e61e049f254bd/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:38.365775 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25abd28d-e040-48f9-a635-5582034e0591 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:38.409716 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9d478c19-8d0f-4079-ad05-1f8ea0f145e0 admin admin] The token cache is being invalidate because project 3f3b2caf25394630b17da048591d2fe8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:26:38.415149 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9d478c19-8d0f-4079-ad05-1f8ea0f145e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:38.424558 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9d478c19-8d0f-4079-ad05-1f8ea0f145e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:38.430513 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 901/1817] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:26:37 2022] DELETE /identity/v3/projects/3f3b2caf25394630b17da048591d2fe8 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:38.430513 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d478c19-8d0f-4079-ad05-1f8ea0f145e0 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:38.432148 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-9d478c19-8d0f-4079-ad05-1f8ea0f145e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:38.466534 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:38.532150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d478c19-8d0f-4079-ad05-1f8ea0f145e0 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:38.600530 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25abd28d-e040-48f9-a635-5582034e0591 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:38.609623 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d787d80b-61df-41f7-98bb-8b97a7eb1a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:38.610434 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787d80b-61df-41f7-98bb-8b97a7eb1a41 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:38.610830 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787d80b-61df-41f7-98bb-8b97a7eb1a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:38.611189 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787d80b-61df-41f7-98bb-8b97a7eb1a41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:38.612448 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d787d80b-61df-41f7-98bb-8b97a7eb1a41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:38.614408 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d787d80b-61df-41f7-98bb-8b97a7eb1a41 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:38.687613 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701fc171-9c56-4522-b0e5-1c7a25834dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:38.688718 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fc171-9c56-4522-b0e5-1c7a25834dc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:38.689312 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fc171-9c56-4522-b0e5-1c7a25834dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:38.689679 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fc171-9c56-4522-b0e5-1c7a25834dc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:38.690432 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701fc171-9c56-4522-b0e5-1c7a25834dc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:38.696661 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701fc171-9c56-4522-b0e5-1c7a25834dc3 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:38.711990 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 902/1818] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:38.712962 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701fc171-9c56-4522-b0e5-1c7a25834dc3 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:38.720017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-701fc171-9c56-4522-b0e5-1c7a25834dc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:38.746388 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701fc171-9c56-4522-b0e5-1c7a25834dc3 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:38.753443 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e984ff5e-1ce1-4cd9-a114-45657b8568c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:38.757382 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984ff5e-1ce1-4cd9-a114-45657b8568c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:38.757382 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984ff5e-1ce1-4cd9-a114-45657b8568c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:38.757382 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984ff5e-1ce1-4cd9-a114-45657b8568c3 admin admin] PATH_INFO: `/v3/users/7128398b42f64c64b3b69357133230a4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:38.757382 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e984ff5e-1ce1-4cd9-a114-45657b8568c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7128398b42f64c64b3b69357133230a4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:38.759474 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e984ff5e-1ce1-4cd9-a114-45657b8568c3 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:38.882767 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 917/1819] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:26:38 2022] POST /identity/v3/projects => generated 407 bytes in 521 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:38.884746 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d787d80b-61df-41f7-98bb-8b97a7eb1a41 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:39.029542 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e984ff5e-1ce1-4cd9-a114-45657b8568c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7128398b42f64c64b3b69357133230a4'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:39.056597 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d787d80b-61df-41f7-98bb-8b97a7eb1a41 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:39.062595 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e984ff5e-1ce1-4cd9-a114-45657b8568c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7128398b42f64c64b3b69357133230a4'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:39.064722 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9d1dc-3440-4c62-ba4f-9775eb75d5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:39.065958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9d1dc-3440-4c62-ba4f-9775eb75d5b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:39.066251 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9d1dc-3440-4c62-ba4f-9775eb75d5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:39.066758 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9d1dc-3440-4c62-ba4f-9775eb75d5b0 admin admin] PATH_INFO: `/v3/projects/412e6396ca0641379f1f09e5ce1fb2bb/users/7bcf963d78854e8a892e61e049f254bd/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:39.070835 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9d1dc-3440-4c62-ba4f-9775eb75d5b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=412e6396ca0641379f1f09e5ce1fb2bb, user_id=7bcf963d78854e8a892e61e049f254bd, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:39.074706 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9d1dc-3440-4c62-ba4f-9775eb75d5b0 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:39.089620 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e984ff5e-1ce1-4cd9-a114-45657b8568c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7128398b42f64c64b3b69357133230a4'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:39.090896 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e984ff5e-1ce1-4cd9-a114-45657b8568c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7128398b42f64c64b3b69357133230a4'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:39.092077 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e984ff5e-1ce1-4cd9-a114-45657b8568c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7128398b42f64c64b3b69357133230a4'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:39.106409 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 918/1820] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:38 2022] PUT /identity/v3/projects/412e6396ca0641379f1f09e5ce1fb2bb/users/7bcf963d78854e8a892e61e049f254bd/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:39.106886 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9d1dc-3440-4c62-ba4f-9775eb75d5b0 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:39.109106 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-21a9d1dc-3440-4c62-ba4f-9775eb75d5b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:39.251224 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9d1dc-3440-4c62-ba4f-9775eb75d5b0 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:39.261701 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175e330-af49-4736-9716-95cd898663a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:39.265448 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175e330-af49-4736-9716-95cd898663a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:39.265448 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175e330-af49-4736-9716-95cd898663a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:39.265448 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175e330-af49-4736-9716-95cd898663a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:39.265448 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b175e330-af49-4736-9716-95cd898663a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:39.265789 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 903/1821] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:26:38 2022] DELETE /identity/v3/users/7128398b42f64c64b3b69357133230a4 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:39.268307 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b175e330-af49-4736-9716-95cd898663a2 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:39.433611 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 919/1822] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:26:39 2022] POST /identity/v3/users => generated 304 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:39.434926 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e984ff5e-1ce1-4cd9-a114-45657b8568c3 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:39.465871 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:39.611850 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e984ff5e-1ce1-4cd9-a114-45657b8568c3 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:39.618724 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa3e21-b424-4345-b7a4-089cbb9d7c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:39.621812 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa3e21-b424-4345-b7a4-089cbb9d7c25 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:39.621812 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa3e21-b424-4345-b7a4-089cbb9d7c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:39.621812 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa3e21-b424-4345-b7a4-089cbb9d7c25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:39.621812 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa3e21-b424-4345-b7a4-089cbb9d7c25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:39.623592 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa3e21-b424-4345-b7a4-089cbb9d7c25 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:39.642080 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 904/1823] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 06:26:39.644156 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175e330-af49-4736-9716-95cd898663a2 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:39.661790 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175e330-af49-4736-9716-95cd898663a2 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:39.680559 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ae3c0-051e-40fd-b53d-3818b86f39e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:39.682814 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ae3c0-051e-40fd-b53d-3818b86f39e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:39.683964 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ae3c0-051e-40fd-b53d-3818b86f39e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:39.684676 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ae3c0-051e-40fd-b53d-3818b86f39e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:39.690227 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ae3c0-051e-40fd-b53d-3818b86f39e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:39.690227 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ae3c0-051e-40fd-b53d-3818b86f39e9 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:39.719480 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 920/1824] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:39.725195 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa3e21-b424-4345-b7a4-089cbb9d7c25 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:39.760401 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa3e21-b424-4345-b7a4-089cbb9d7c25 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:39.789316 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11892dcf-cea3-45b4-8786-ca3ff117e1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:39.790643 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11892dcf-cea3-45b4-8786-ca3ff117e1f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:39.790643 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11892dcf-cea3-45b4-8786-ca3ff117e1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:39.790939 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11892dcf-cea3-45b4-8786-ca3ff117e1f3 admin admin] PATH_INFO: `/v3/projects/7ac94c43103f4046a476a88d8b7028c8/users/6eb414ca9a5448c89b52f56250548034/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:39.823968 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11892dcf-cea3-45b4-8786-ca3ff117e1f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ac94c43103f4046a476a88d8b7028c8, user_id=6eb414ca9a5448c89b52f56250548034, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:39.827056 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11892dcf-cea3-45b4-8786-ca3ff117e1f3 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:39.962379 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 905/1825] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:39 2022] PUT /identity/v3/projects/7ac94c43103f4046a476a88d8b7028c8/users/6eb414ca9a5448c89b52f56250548034/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:39.966176 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a554960-6add-4c25-a0e9-4f70199bde36 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:39.966522 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a554960-6add-4c25-a0e9-4f70199bde36 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:39.966818 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a554960-6add-4c25-a0e9-4f70199bde36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:40.088956 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-3a554960-6add-4c25-a0e9-4f70199bde36 None None] MFA Rules not processed for user `6eb414ca9a5448c89b52f56250548034`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:40.092269 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-3a554960-6add-4c25-a0e9-4f70199bde36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:40.231208 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 921/1826] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:39 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:26:40.233394 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11892dcf-cea3-45b4-8786-ca3ff117e1f3 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:40.367419 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:40.405344 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11892dcf-cea3-45b4-8786-ca3ff117e1f3 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:40.413181 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee01b648-ece1-4053-b582-78b617b6fdd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:40.414682 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01b648-ece1-4053-b582-78b617b6fdd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:40.415656 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01b648-ece1-4053-b582-78b617b6fdd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:40.416924 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01b648-ece1-4053-b582-78b617b6fdd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:40.424408 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee01b648-ece1-4053-b582-78b617b6fdd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:40.424408 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee01b648-ece1-4053-b582-78b617b6fdd5 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:40.576335 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a554960-6add-4c25-a0e9-4f70199bde36 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:40.588928 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 906/1827] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:26:40 2022] POST /identity/v3/projects => generated 407 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:40.604205 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a554960-6add-4c25-a0e9-4f70199bde36 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:40.727382 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2659721c-ca92-45a0-96da-e8f5e59075fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:40.731731 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659721c-ca92-45a0-96da-e8f5e59075fc admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:40.731731 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659721c-ca92-45a0-96da-e8f5e59075fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:40.731731 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659721c-ca92-45a0-96da-e8f5e59075fc admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:40.731731 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2659721c-ca92-45a0-96da-e8f5e59075fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:40.732633 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2659721c-ca92-45a0-96da-e8f5e59075fc admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:40.795037 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee01b648-ece1-4053-b582-78b617b6fdd5 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:40.798399 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ee01b648-ece1-4053-b582-78b617b6fdd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:40.815562 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee01b648-ece1-4053-b582-78b617b6fdd5 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:40.824947 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91923ab6-cbd0-40a5-a9da-957928e94e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:40.826320 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91923ab6-cbd0-40a5-a9da-957928e94e60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:40.826885 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91923ab6-cbd0-40a5-a9da-957928e94e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:40.827676 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91923ab6-cbd0-40a5-a9da-957928e94e60 admin admin] PATH_INFO: `/v3/projects/1a86a5020fe34e098377d2017b59eace` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:40.829951 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91923ab6-cbd0-40a5-a9da-957928e94e60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a86a5020fe34e098377d2017b59eace)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:40.833424 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91923ab6-cbd0-40a5-a9da-957928e94e60 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:40.840202 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 922/1828] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:26:40 2022] POST /identity/v3/users => generated 310 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:26:40.849049 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90e736-0582-405f-a958-1429326c9d0d None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:40.849753 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90e736-0582-405f-a958-1429326c9d0d None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:40.869864 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90e736-0582-405f-a958-1429326c9d0d None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:40.877030 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 923/1829] 10.222.0.40 () {54 vars in 921 bytes} [Mon Jul 18 06:26:40 2022] GET /identity => generated 270 bytes in 32 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:26:40.878811 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90e736-0582-405f-a958-1429326c9d0d None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:40.928093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90e736-0582-405f-a958-1429326c9d0d None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:40.934261 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea739c17-52f6-40a9-b894-e00086ba8993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:40.935087 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea739c17-52f6-40a9-b894-e00086ba8993 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:40.935421 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea739c17-52f6-40a9-b894-e00086ba8993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:40.935706 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea739c17-52f6-40a9-b894-e00086ba8993 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:40.936515 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea739c17-52f6-40a9-b894-e00086ba8993 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:40.942788 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea739c17-52f6-40a9-b894-e00086ba8993 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:40.953327 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 924/1830] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:40.955041 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f14b3a-a251-4f3a-9330-7c0b9fe41388 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:40.955335 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f14b3a-a251-4f3a-9330-7c0b9fe41388 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:40.955617 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f14b3a-a251-4f3a-9330-7c0b9fe41388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:40.989950 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-93f14b3a-a251-4f3a-9330-7c0b9fe41388 None None] MFA Rules not processed for user `f92d948810ea4241810473506b654f7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:40.994602 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-93f14b3a-a251-4f3a-9330-7c0b9fe41388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:41.037447 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-91923ab6-cbd0-40a5-a9da-957928e94e60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a86a5020fe34e098377d2017b59eace', 'request_id': 'req-91923ab6-cbd0-40a5-a9da-957928e94e60'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:41.049451 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-91923ab6-cbd0-40a5-a9da-957928e94e60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a86a5020fe34e098377d2017b59eace', 'request_id': 'req-91923ab6-cbd0-40a5-a9da-957928e94e60'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:41.142833 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-91923ab6-cbd0-40a5-a9da-957928e94e60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a86a5020fe34e098377d2017b59eace', 'request_id': 'req-91923ab6-cbd0-40a5-a9da-957928e94e60'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:41.168064 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-91923ab6-cbd0-40a5-a9da-957928e94e60 admin admin] The token cache is being invalidate because project 1a86a5020fe34e098377d2017b59eace was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:26:41.168342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-91923ab6-cbd0-40a5-a9da-957928e94e60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:41.169811 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-91923ab6-cbd0-40a5-a9da-957928e94e60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:41.175166 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 907/1831] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:26:40 2022] DELETE /identity/v3/projects/1a86a5020fe34e098377d2017b59eace => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:41.180540 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91923ab6-cbd0-40a5-a9da-957928e94e60 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:41.185524 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-91923ab6-cbd0-40a5-a9da-957928e94e60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:41.263997 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 925/1832] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Jul 18 06:26:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:41.267343 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ea3d5-dc5e-44e5-a99c-9b0fc0ec01ab None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:41.272200 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ea3d5-dc5e-44e5-a99c-9b0fc0ec01ab None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:41.273014 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ea3d5-dc5e-44e5-a99c-9b0fc0ec01ab None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:41.280096 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 926/1833] 10.222.0.40 () {54 vars in 921 bytes} [Mon Jul 18 06:26:41 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:26:41.286979 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b5d1c-f06f-4e70-960f-5db68915ea74 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:41.287986 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b5d1c-f06f-4e70-960f-5db68915ea74 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:41.288777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b5d1c-f06f-4e70-960f-5db68915ea74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:41.330865 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-e64b5d1c-f06f-4e70-960f-5db68915ea74 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:41.333203 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e64b5d1c-f06f-4e70-960f-5db68915ea74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:41.362449 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91923ab6-cbd0-40a5-a9da-957928e94e60 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:41.362598 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 927/1834] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:41.365241 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23583a5c-1faf-4632-bb0c-5feb1095c787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:41.366568 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23583a5c-1faf-4632-bb0c-5feb1095c787 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:41.367088 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23583a5c-1faf-4632-bb0c-5feb1095c787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:41.367589 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23583a5c-1faf-4632-bb0c-5feb1095c787 admin admin] PATH_INFO: `/v3/projects/b998419218344ecaa336c7476cc8cbaf/users/53bce8abe1774413a86ba2411c4603fe/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:41.378241 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf9c03-531a-449e-a5cc-b8db0a848ae6 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:41.380002 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf9c03-531a-449e-a5cc-b8db0a848ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:41.380890 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf9c03-531a-449e-a5cc-b8db0a848ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:41.388797 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:41.396354 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23583a5c-1faf-4632-bb0c-5feb1095c787 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b998419218344ecaa336c7476cc8cbaf, user_id=53bce8abe1774413a86ba2411c4603fe, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:41.400582 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23583a5c-1faf-4632-bb0c-5feb1095c787 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:41.417876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-ffaf9c03-531a-449e-a5cc-b8db0a848ae6 None None] MFA Rules not processed for user `f92d948810ea4241810473506b654f7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:41.420547 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ffaf9c03-531a-449e-a5cc-b8db0a848ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:41.444455 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 928/1835] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Jul 18 06:26:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:41.453078 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b47f1b-6b9b-400a-a978-922b08e8a54a None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:41.455208 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b47f1b-6b9b-400a-a978-922b08e8a54a None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:41.456149 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b47f1b-6b9b-400a-a978-922b08e8a54a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:41.501417 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-26b47f1b-6b9b-400a-a978-922b08e8a54a None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:41.503988 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-26b47f1b-6b9b-400a-a978-922b08e8a54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:41.526472 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 908/1836] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:41 2022] PUT /identity/v3/projects/b998419218344ecaa336c7476cc8cbaf/users/53bce8abe1774413a86ba2411c4603fe/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:41.526472 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23583a5c-1faf-4632-bb0c-5feb1095c787 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:41.527225 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-23583a5c-1faf-4632-bb0c-5feb1095c787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:41.549672 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:41.550486 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 929/1837] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:41.556007 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b47f1b-6b9b-400a-a978-922b08e8a54a None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:41.559412 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-26b47f1b-6b9b-400a-a978-922b08e8a54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:41.588907 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b47f1b-6b9b-400a-a978-922b08e8a54a None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:41.627245 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23583a5c-1faf-4632-bb0c-5feb1095c787 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:41.654701 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e2af89-64f9-4172-89c8-a01aecf9509f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:41.654701 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e2af89-64f9-4172-89c8-a01aecf9509f service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:41.654701 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e2af89-64f9-4172-89c8-a01aecf9509f service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:41.654701 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e2af89-64f9-4172-89c8-a01aecf9509f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:41.655290 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e2af89-64f9-4172-89c8-a01aecf9509f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:41.655290 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e2af89-64f9-4172-89c8-a01aecf9509f service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:41.660021 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 909/1838] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:41.661135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e2af89-64f9-4172-89c8-a01aecf9509f service nova] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:41.698363 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b387ce0b-181f-471a-a17f-0cdea1205c9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:41.699087 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387ce0b-181f-471a-a17f-0cdea1205c9a service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:41.700721 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387ce0b-181f-471a-a17f-0cdea1205c9a service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:41.700721 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387ce0b-181f-471a-a17f-0cdea1205c9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:41.711694 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b387ce0b-181f-471a-a17f-0cdea1205c9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:41.713454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b387ce0b-181f-471a-a17f-0cdea1205c9a service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:41.726280 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 930/1839] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:41.726908 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b387ce0b-181f-471a-a17f-0cdea1205c9a service nova] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:41.788582 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e2af89-64f9-4172-89c8-a01aecf9509f service nova] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:41.794691 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54aa7c8-80d5-4bc1-8659-49f31be78ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:41.795650 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54aa7c8-80d5-4bc1-8659-49f31be78ab2 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:41.796251 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54aa7c8-80d5-4bc1-8659-49f31be78ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:41.796251 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54aa7c8-80d5-4bc1-8659-49f31be78ab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:41.797030 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54aa7c8-80d5-4bc1-8659-49f31be78ab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:41.799040 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54aa7c8-80d5-4bc1-8659-49f31be78ab2 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:41.817794 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 910/1840] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:41.825570 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b387ce0b-181f-471a-a17f-0cdea1205c9a service nova] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:41.832086 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54aa7c8-80d5-4bc1-8659-49f31be78ab2 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:41.833534 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9a4b0-ee3a-4e42-8070-9b75c338f274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:41.838944 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9a4b0-ee3a-4e42-8070-9b75c338f274 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:41.839711 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9a4b0-ee3a-4e42-8070-9b75c338f274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:41.840184 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9a4b0-ee3a-4e42-8070-9b75c338f274 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:41.841182 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d9a4b0-ee3a-4e42-8070-9b75c338f274 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:26:41.841841 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d9a4b0-ee3a-4e42-8070-9b75c338f274 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:41.844086 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d9a4b0-ee3a-4e42-8070-9b75c338f274 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:41.855045 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54aa7c8-80d5-4bc1-8659-49f31be78ab2 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:41.861283 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 931/1841] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:26:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:26:41.866668 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9a4b0-ee3a-4e42-8070-9b75c338f274 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:41.868302 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0863146-c91d-4ba6-acfc-959d670f50de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:41.869269 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0863146-c91d-4ba6-acfc-959d670f50de admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:41.869550 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0863146-c91d-4ba6-acfc-959d670f50de admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:41.869825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0863146-c91d-4ba6-acfc-959d670f50de admin admin] PATH_INFO: `/v3/projects/b998419218344ecaa336c7476cc8cbaf/users/53bce8abe1774413a86ba2411c4603fe/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:41.876666 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0863146-c91d-4ba6-acfc-959d670f50de admin admin] RBAC: Authorizing `identity:create_grant(project_id=b998419218344ecaa336c7476cc8cbaf, user_id=53bce8abe1774413a86ba2411c4603fe, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:41.880243 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0863146-c91d-4ba6-acfc-959d670f50de admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:41.885322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9a4b0-ee3a-4e42-8070-9b75c338f274 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:41.890277 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f967e-1335-4730-92c2-caa14d590f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:41.891334 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f967e-1335-4730-92c2-caa14d590f3e admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:41.891632 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f967e-1335-4730-92c2-caa14d590f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:41.892263 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f967e-1335-4730-92c2-caa14d590f3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:41.894946 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f967e-1335-4730-92c2-caa14d590f3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:41.897591 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f967e-1335-4730-92c2-caa14d590f3e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:41.916622 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 911/1842] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:41 2022] PUT /identity/v3/projects/b998419218344ecaa336c7476cc8cbaf/users/53bce8abe1774413a86ba2411c4603fe/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:26:41.922463 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08c669-a740-44af-bfc5-376289ae09f2 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:41.922463 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08c669-a740-44af-bfc5-376289ae09f2 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:41.923175 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08c669-a740-44af-bfc5-376289ae09f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:42.008991 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-0e08c669-a740-44af-bfc5-376289ae09f2 None None] MFA Rules not processed for user `7bcf963d78854e8a892e61e049f254bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:42.017613 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-0e08c669-a740-44af-bfc5-376289ae09f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:42.064780 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 932/1843] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:26:41 2022] POST /identity/v3/projects => generated 377 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:42.067599 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91adcac-b636-479a-8843-c80810a88d01 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:42.068385 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91adcac-b636-479a-8843-c80810a88d01 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:42.075264 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91adcac-b636-479a-8843-c80810a88d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:42.123847 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-d91adcac-b636-479a-8843-c80810a88d01 None None] MFA Rules not processed for user `53bce8abe1774413a86ba2411c4603fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:42.125467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d91adcac-b636-479a-8843-c80810a88d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:42.197754 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 912/1844] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:41 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:26:42.199175 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08c669-a740-44af-bfc5-376289ae09f2 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:42.277521 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 933/1845] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:42 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:42.306450 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91adcac-b636-479a-8843-c80810a88d01 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:42.312240 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d91adcac-b636-479a-8843-c80810a88d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:42.374559 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08c669-a740-44af-bfc5-376289ae09f2 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:42.466020 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91adcac-b636-479a-8843-c80810a88d01 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:42.472408 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f5808f-bdb5-44b5-8f8d-e24f56c781e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:42.473753 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5808f-bdb5-44b5-8f8d-e24f56c781e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:42.474093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5808f-bdb5-44b5-8f8d-e24f56c781e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:42.474585 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5808f-bdb5-44b5-8f8d-e24f56c781e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:42.479005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa585d-3a44-4d37-a195-ab2eea16a7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:42.479700 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa585d-3a44-4d37-a195-ab2eea16a7f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:42.479990 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa585d-3a44-4d37-a195-ab2eea16a7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:42.480272 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa585d-3a44-4d37-a195-ab2eea16a7f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:42.481021 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fa585d-3a44-4d37-a195-ab2eea16a7f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:42.482992 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fa585d-3a44-4d37-a195-ab2eea16a7f3 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:42.490675 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f5808f-bdb5-44b5-8f8d-e24f56c781e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:42.492673 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f5808f-bdb5-44b5-8f8d-e24f56c781e2 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:42.505705 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 934/1846] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:42 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 06:26:42.588514 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 913/1847] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:26:42 2022] POST /identity/v3/users => generated 289 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:42.594184 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f5808f-bdb5-44b5-8f8d-e24f56c781e2 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:42.618028 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f5808f-bdb5-44b5-8f8d-e24f56c781e2 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:42.625458 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db83df2-0d11-4037-9e28-9404c82a32e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:42.633617 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db83df2-0d11-4037-9e28-9404c82a32e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:42.633617 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db83df2-0d11-4037-9e28-9404c82a32e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:42.633617 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db83df2-0d11-4037-9e28-9404c82a32e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:42.633617 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db83df2-0d11-4037-9e28-9404c82a32e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:42.633617 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db83df2-0d11-4037-9e28-9404c82a32e2 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:42.642537 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 935/1848] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:42.646772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa585d-3a44-4d37-a195-ab2eea16a7f3 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:42.657785 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa585d-3a44-4d37-a195-ab2eea16a7f3 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:42.663197 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b78fe95-e80c-4251-8654-5cd6afed3a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:42.663722 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b78fe95-e80c-4251-8654-5cd6afed3a0b admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:42.664594 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b78fe95-e80c-4251-8654-5cd6afed3a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:42.664594 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b78fe95-e80c-4251-8654-5cd6afed3a0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:42.665118 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b78fe95-e80c-4251-8654-5cd6afed3a0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:42.668061 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b78fe95-e80c-4251-8654-5cd6afed3a0b admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:42.679714 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 914/1849] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:42.683917 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db83df2-0d11-4037-9e28-9404c82a32e2 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:42.695756 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db83df2-0d11-4037-9e28-9404c82a32e2 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:42.701079 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa442095-2720-4ec3-812a-b3295a2a47fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:42.702006 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa442095-2720-4ec3-812a-b3295a2a47fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:42.702297 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa442095-2720-4ec3-812a-b3295a2a47fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:42.702582 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa442095-2720-4ec3-812a-b3295a2a47fe admin admin] PATH_INFO: `/v3/projects/fcc46a296c5b4bd79fed7a261b121aa2/users/9c8cc43dac064ef4a3da462459a20d50/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:42.721804 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa442095-2720-4ec3-812a-b3295a2a47fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcc46a296c5b4bd79fed7a261b121aa2, user_id=9c8cc43dac064ef4a3da462459a20d50, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:42.725593 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa442095-2720-4ec3-812a-b3295a2a47fe admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:42.729894 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:42.939133 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 936/1850] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:42 2022] PUT /identity/v3/projects/fcc46a296c5b4bd79fed7a261b121aa2/users/9c8cc43dac064ef4a3da462459a20d50/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:42.951807 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8b63a-da2f-4aa8-aff8-cc4c03683294 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:42.952383 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8b63a-da2f-4aa8-aff8-cc4c03683294 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:42.952939 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8b63a-da2f-4aa8-aff8-cc4c03683294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:42.997140 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-60f8b63a-da2f-4aa8-aff8-cc4c03683294 None None] MFA Rules not processed for user `9c8cc43dac064ef4a3da462459a20d50`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:42.998743 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-60f8b63a-da2f-4aa8-aff8-cc4c03683294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:43.035627 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa442095-2720-4ec3-812a-b3295a2a47fe admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:43.136913 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 915/1851] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:43.141380 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f8b63a-da2f-4aa8-aff8-cc4c03683294 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:43.164302 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa442095-2720-4ec3-812a-b3295a2a47fe admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:43.173657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754efae8-a609-4f76-89bf-332a99e714ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:43.175168 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754efae8-a609-4f76-89bf-332a99e714ba service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:43.176215 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754efae8-a609-4f76-89bf-332a99e714ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:43.176949 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754efae8-a609-4f76-89bf-332a99e714ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:43.180208 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-754efae8-a609-4f76-89bf-332a99e714ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:43.241523 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:43.248371 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f8b63a-da2f-4aa8-aff8-cc4c03683294 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:43.256092 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c7d0b5-45ef-46da-abdc-c63785c16fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:43.256993 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c7d0b5-45ef-46da-abdc-c63785c16fbc admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:43.257468 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c7d0b5-45ef-46da-abdc-c63785c16fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:43.257946 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c7d0b5-45ef-46da-abdc-c63785c16fbc admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:43.258804 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c7d0b5-45ef-46da-abdc-c63785c16fbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:43.262027 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c7d0b5-45ef-46da-abdc-c63785c16fbc admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:43.281440 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754efae8-a609-4f76-89bf-332a99e714ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:43.283032 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754efae8-a609-4f76-89bf-332a99e714ba service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:43.295694 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 937/1852] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:43 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:43.296910 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8060cce3-a67b-456d-93a4-3b1881066d98 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:43.298437 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8060cce3-a67b-456d-93a4-3b1881066d98 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:43.298437 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8060cce3-a67b-456d-93a4-3b1881066d98 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:43.299054 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 938/1853] 10.222.0.40 () {54 vars in 921 bytes} [Mon Jul 18 06:26:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:26:43.309380 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504da46-4f28-423b-878c-9c9af2202d2a None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:43.310026 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504da46-4f28-423b-878c-9c9af2202d2a None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:43.311749 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504da46-4f28-423b-878c-9c9af2202d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:43.367688 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-c504da46-4f28-423b-878c-9c9af2202d2a None None] MFA Rules not processed for user `a3560534427544cdabfd0ba702a0a165`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:43.393758 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c504da46-4f28-423b-878c-9c9af2202d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:43.414686 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 916/1854] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:26:43 2022] POST /identity/v3/projects => generated 377 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:26:43.424691 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c7d0b5-45ef-46da-abdc-c63785c16fbc admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:43.542448 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 939/1855] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Jul 18 06:26:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:43.594119 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c7d0b5-45ef-46da-abdc-c63785c16fbc admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:43.600641 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c504da46-4f28-423b-878c-9c9af2202d2a None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:43.604162 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098fa729-cbc0-44d1-957c-d52921e23361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:43.605200 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098fa729-cbc0-44d1-957c-d52921e23361 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:43.605803 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098fa729-cbc0-44d1-957c-d52921e23361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:43.606438 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098fa729-cbc0-44d1-957c-d52921e23361 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:43.607687 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c504da46-4f28-423b-878c-9c9af2202d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:43.608972 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098fa729-cbc0-44d1-957c-d52921e23361 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:43.613076 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098fa729-cbc0-44d1-957c-d52921e23361 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:43.637954 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c504da46-4f28-423b-878c-9c9af2202d2a None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:43.651918 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ce4e51-9784-4bd2-8455-c94b308af868 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3560534427544cdabfd0ba702a0a165', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:43.653766 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce4e51-9784-4bd2-8455-c94b308af868 service placement] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:43.654394 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce4e51-9784-4bd2-8455-c94b308af868 service placement] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:43.655921 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce4e51-9784-4bd2-8455-c94b308af868 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:43.669537 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ce4e51-9784-4bd2-8455-c94b308af868 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:43.671946 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ce4e51-9784-4bd2-8455-c94b308af868 service placement] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:43.685777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 940/1856] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:43.712110 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 917/1857] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:26:43 2022] POST /identity/v3/users => generated 295 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:43.716851 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ce4e51-9784-4bd2-8455-c94b308af868 service placement] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:43.745353 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ce4e51-9784-4bd2-8455-c94b308af868 service placement] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:43.756662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f66ad7-0a70-4c2d-bc84-9692b7204b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:43.758242 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f66ad7-0a70-4c2d-bc84-9692b7204b9e admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:43.759008 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f66ad7-0a70-4c2d-bc84-9692b7204b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:43.759695 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f66ad7-0a70-4c2d-bc84-9692b7204b9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:43.761812 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f66ad7-0a70-4c2d-bc84-9692b7204b9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:43.765535 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f66ad7-0a70-4c2d-bc84-9692b7204b9e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:43.792790 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 941/1858] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:43 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 06:26:43.797788 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098fa729-cbc0-44d1-957c-d52921e23361 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:43.812153 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098fa729-cbc0-44d1-957c-d52921e23361 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:43.818879 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7ccec6-f48b-4ad1-a8b6-9b6c5efe5935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:43.819932 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ccec6-f48b-4ad1-a8b6-9b6c5efe5935 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:43.820538 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ccec6-f48b-4ad1-a8b6-9b6c5efe5935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:43.821102 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ccec6-f48b-4ad1-a8b6-9b6c5efe5935 admin admin] PATH_INFO: `/v3/projects/9f03abcc62d64a489ca80bb90ca7fbdc/users/83f9f740cdd9480da49b5a6a07552b4f/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:43.845560 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7ccec6-f48b-4ad1-a8b6-9b6c5efe5935 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f03abcc62d64a489ca80bb90ca7fbdc, user_id=83f9f740cdd9480da49b5a6a07552b4f, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:43.851860 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7ccec6-f48b-4ad1-a8b6-9b6c5efe5935 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:43.974400 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 918/1859] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:43 2022] PUT /identity/v3/projects/9f03abcc62d64a489ca80bb90ca7fbdc/users/83f9f740cdd9480da49b5a6a07552b4f/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:43.979462 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f66ad7-0a70-4c2d-bc84-9692b7204b9e admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:43.992544 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f66ad7-0a70-4c2d-bc84-9692b7204b9e admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:44.085652 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4294d54a-b818-4deb-b638-aa70759a28c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:44.086354 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294d54a-b818-4deb-b638-aa70759a28c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:44.086827 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294d54a-b818-4deb-b638-aa70759a28c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:44.087144 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294d54a-b818-4deb-b638-aa70759a28c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:44.087928 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4294d54a-b818-4deb-b638-aa70759a28c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:44.089677 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4294d54a-b818-4deb-b638-aa70759a28c1 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:44.104099 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 942/1860] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:43 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:44.110061 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7ccec6-f48b-4ad1-a8b6-9b6c5efe5935 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:44.130928 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7ccec6-f48b-4ad1-a8b6-9b6c5efe5935 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:44.138278 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:44.142114 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281b24b-09ae-4f07-a3de-93bafe24126e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:44.142114 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281b24b-09ae-4f07-a3de-93bafe24126e admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:44.142114 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281b24b-09ae-4f07-a3de-93bafe24126e admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:44.142114 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281b24b-09ae-4f07-a3de-93bafe24126e admin admin] PATH_INFO: `/v3/projects/9f03abcc62d64a489ca80bb90ca7fbdc/users/83f9f740cdd9480da49b5a6a07552b4f/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:44.147362 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0281b24b-09ae-4f07-a3de-93bafe24126e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f03abcc62d64a489ca80bb90ca7fbdc, user_id=83f9f740cdd9480da49b5a6a07552b4f, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:44.150317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0281b24b-09ae-4f07-a3de-93bafe24126e admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:44.164746 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 919/1861] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:44 2022] PUT /identity/v3/projects/9f03abcc62d64a489ca80bb90ca7fbdc/users/83f9f740cdd9480da49b5a6a07552b4f/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:44.175361 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23a0db-a850-4d9d-9912-d54ee1686ca8 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:44.175794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23a0db-a850-4d9d-9912-d54ee1686ca8 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:44.176383 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23a0db-a850-4d9d-9912-d54ee1686ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:44.212732 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-ad23a0db-a850-4d9d-9912-d54ee1686ca8 None None] MFA Rules not processed for user `83f9f740cdd9480da49b5a6a07552b4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:44.214006 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ad23a0db-a850-4d9d-9912-d54ee1686ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:44.343924 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 943/1862] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:44 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:44.358991 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281b24b-09ae-4f07-a3de-93bafe24126e admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:44.361891 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-0281b24b-09ae-4f07-a3de-93bafe24126e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:44.471642 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281b24b-09ae-4f07-a3de-93bafe24126e admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:44.480214 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfffc863-6606-4fde-8e34-b1d335e3ce1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:44.480984 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfffc863-6606-4fde-8e34-b1d335e3ce1a service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:44.481270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfffc863-6606-4fde-8e34-b1d335e3ce1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:44.481552 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfffc863-6606-4fde-8e34-b1d335e3ce1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:44.573607 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfffc863-6606-4fde-8e34-b1d335e3ce1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:44.575407 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfffc863-6606-4fde-8e34-b1d335e3ce1a service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:44.591165 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 920/1863] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:44.673971 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23a0db-a850-4d9d-9912-d54ee1686ca8 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:44.677736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ad23a0db-a850-4d9d-9912-d54ee1686ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:44.875838 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23a0db-a850-4d9d-9912-d54ee1686ca8 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:44.885694 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22ff43f-5b67-445d-bcbf-6b6b723602bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:44.886720 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ff43f-5b67-445d-bcbf-6b6b723602bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:44.887176 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ff43f-5b67-445d-bcbf-6b6b723602bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:44.889185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ff43f-5b67-445d-bcbf-6b6b723602bb admin admin] PATH_INFO: `/v3/users/ffc85a80354d4d1c8cc066c519ea29a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:44.891348 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22ff43f-5b67-445d-bcbf-6b6b723602bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffc85a80354d4d1c8cc066c519ea29a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:44.893928 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22ff43f-5b67-445d-bcbf-6b6b723602bb admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:45.168381 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-f22ff43f-5b67-445d-bcbf-6b6b723602bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffc85a80354d4d1c8cc066c519ea29a7'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:45.182691 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-f22ff43f-5b67-445d-bcbf-6b6b723602bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffc85a80354d4d1c8cc066c519ea29a7'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:45.184085 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-f22ff43f-5b67-445d-bcbf-6b6b723602bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffc85a80354d4d1c8cc066c519ea29a7'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:45.184799 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-f22ff43f-5b67-445d-bcbf-6b6b723602bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffc85a80354d4d1c8cc066c519ea29a7'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:45.205678 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-f22ff43f-5b67-445d-bcbf-6b6b723602bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffc85a80354d4d1c8cc066c519ea29a7'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:45.342329 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 944/1864] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:26:44 2022] DELETE /identity/v3/users/ffc85a80354d4d1c8cc066c519ea29a7 => generated 0 bytes in 669 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:45.510412 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:45.783069 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:46.152163 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfffc863-6606-4fde-8e34-b1d335e3ce1a service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:46.175914 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfffc863-6606-4fde-8e34-b1d335e3ce1a service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:46.334273 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5d861-8f50-431d-9786-c9329b827b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:46.338875 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5d861-8f50-431d-9786-c9329b827b31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:46.338875 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5d861-8f50-431d-9786-c9329b827b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:46.338875 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5d861-8f50-431d-9786-c9329b827b31 admin admin] PATH_INFO: `/v3/projects/0db668553be14231aea54d7d51c0def6` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:46.338875 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc5d861-8f50-431d-9786-c9329b827b31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0db668553be14231aea54d7d51c0def6)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:46.343573 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc5d861-8f50-431d-9786-c9329b827b31 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:46.628243 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-ddc5d861-8f50-431d-9786-c9329b827b31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0db668553be14231aea54d7d51c0def6', 'request_id': 'req-ddc5d861-8f50-431d-9786-c9329b827b31'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:46.635653 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-ddc5d861-8f50-431d-9786-c9329b827b31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0db668553be14231aea54d7d51c0def6', 'request_id': 'req-ddc5d861-8f50-431d-9786-c9329b827b31'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:46.730713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-ddc5d861-8f50-431d-9786-c9329b827b31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0db668553be14231aea54d7d51c0def6', 'request_id': 'req-ddc5d861-8f50-431d-9786-c9329b827b31'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:46.751943 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-ddc5d861-8f50-431d-9786-c9329b827b31 admin admin] The token cache is being invalidate because project 0db668553be14231aea54d7d51c0def6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:26:46.752212 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-ddc5d861-8f50-431d-9786-c9329b827b31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:46.758764 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-ddc5d861-8f50-431d-9786-c9329b827b31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:46.765371 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 921/1865] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:26:46 2022] DELETE /identity/v3/projects/0db668553be14231aea54d7d51c0def6 => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:46.767610 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22ff43f-5b67-445d-bcbf-6b6b723602bb admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:46.770200 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-f22ff43f-5b67-445d-bcbf-6b6b723602bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:46.815743 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22ff43f-5b67-445d-bcbf-6b6b723602bb admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:46.964754 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde6e15-aa66-4a11-ac4d-3f00efb87f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:46.970031 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde6e15-aa66-4a11-ac4d-3f00efb87f50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:46.972832 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde6e15-aa66-4a11-ac4d-3f00efb87f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:46.973664 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde6e15-aa66-4a11-ac4d-3f00efb87f50 admin admin] PATH_INFO: `/v3/users/7bcf963d78854e8a892e61e049f254bd` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:46.984839 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbde6e15-aa66-4a11-ac4d-3f00efb87f50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bcf963d78854e8a892e61e049f254bd)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:46.992002 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbde6e15-aa66-4a11-ac4d-3f00efb87f50 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:47.096536 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:47.320500 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-cbde6e15-aa66-4a11-ac4d-3f00efb87f50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bcf963d78854e8a892e61e049f254bd'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:47.346670 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-cbde6e15-aa66-4a11-ac4d-3f00efb87f50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bcf963d78854e8a892e61e049f254bd'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:47.348354 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-cbde6e15-aa66-4a11-ac4d-3f00efb87f50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bcf963d78854e8a892e61e049f254bd'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:47.349519 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-cbde6e15-aa66-4a11-ac4d-3f00efb87f50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bcf963d78854e8a892e61e049f254bd'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:47.376047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-cbde6e15-aa66-4a11-ac4d-3f00efb87f50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bcf963d78854e8a892e61e049f254bd'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:47.412005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:47.499888 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 945/1866] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:26:46 2022] DELETE /identity/v3/users/7bcf963d78854e8a892e61e049f254bd => generated 0 bytes in 728 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:47.950364 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:47.964376 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5d861-8f50-431d-9786-c9329b827b31 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:47.965908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ddc5d861-8f50-431d-9786-c9329b827b31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:48.007613 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5d861-8f50-431d-9786-c9329b827b31 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:48.150917 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dd30ba-3ee9-4dfd-a5f5-47bfe9b0574e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:48.160644 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd30ba-3ee9-4dfd-a5f5-47bfe9b0574e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:48.161790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd30ba-3ee9-4dfd-a5f5-47bfe9b0574e admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:48.162907 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd30ba-3ee9-4dfd-a5f5-47bfe9b0574e admin admin] PATH_INFO: `/v3/projects/412e6396ca0641379f1f09e5ce1fb2bb` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:48.168707 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dd30ba-3ee9-4dfd-a5f5-47bfe9b0574e admin admin] RBAC: Authorizing `identity:delete_project(project_id=412e6396ca0641379f1f09e5ce1fb2bb)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:48.176775 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dd30ba-3ee9-4dfd-a5f5-47bfe9b0574e admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:48.398931 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b9dd30ba-3ee9-4dfd-a5f5-47bfe9b0574e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '412e6396ca0641379f1f09e5ce1fb2bb', 'request_id': 'req-b9dd30ba-3ee9-4dfd-a5f5-47bfe9b0574e'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:48.407509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b9dd30ba-3ee9-4dfd-a5f5-47bfe9b0574e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '412e6396ca0641379f1f09e5ce1fb2bb', 'request_id': 'req-b9dd30ba-3ee9-4dfd-a5f5-47bfe9b0574e'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:48.484321 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b9dd30ba-3ee9-4dfd-a5f5-47bfe9b0574e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '412e6396ca0641379f1f09e5ce1fb2bb', 'request_id': 'req-b9dd30ba-3ee9-4dfd-a5f5-47bfe9b0574e'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:48.527415 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b9dd30ba-3ee9-4dfd-a5f5-47bfe9b0574e admin admin] The token cache is being invalidate because project 412e6396ca0641379f1f09e5ce1fb2bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:26:48.529621 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b9dd30ba-3ee9-4dfd-a5f5-47bfe9b0574e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:48.533668 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b9dd30ba-3ee9-4dfd-a5f5-47bfe9b0574e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:48.538443 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 922/1867] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:26:47 2022] DELETE /identity/v3/projects/412e6396ca0641379f1f09e5ce1fb2bb => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:48.546382 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05342fa-7a62-47e1-b317-295d578a4985 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:48.548103 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05342fa-7a62-47e1-b317-295d578a4985 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:48.548308 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05342fa-7a62-47e1-b317-295d578a4985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:48.584122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-c05342fa-7a62-47e1-b317-295d578a4985 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:48.585582 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c05342fa-7a62-47e1-b317-295d578a4985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:48.726897 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 946/1868] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:48.735154 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79673957-c9af-486e-aad2-f11601da47c1 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:48.735410 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79673957-c9af-486e-aad2-f11601da47c1 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:48.735683 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79673957-c9af-486e-aad2-f11601da47c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:48.796068 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-79673957-c9af-486e-aad2-f11601da47c1 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:48.798340 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-79673957-c9af-486e-aad2-f11601da47c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:48.825904 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 923/1869] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:26:48.830297 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05342fa-7a62-47e1-b317-295d578a4985 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:48.847834 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05342fa-7a62-47e1-b317-295d578a4985 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:48.857467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638336b0-bacf-4c29-882e-de57a5be1c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:48.859697 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638336b0-bacf-4c29-882e-de57a5be1c91 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:48.859697 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638336b0-bacf-4c29-882e-de57a5be1c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:48.859697 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638336b0-bacf-4c29-882e-de57a5be1c91 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:48.861367 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638336b0-bacf-4c29-882e-de57a5be1c91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:26:48.862020 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638336b0-bacf-4c29-882e-de57a5be1c91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:48.868843 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638336b0-bacf-4c29-882e-de57a5be1c91 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:48.906217 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 947/1870] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:26:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:26:48.907547 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79673957-c9af-486e-aad2-f11601da47c1 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:48.919717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79673957-c9af-486e-aad2-f11601da47c1 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:48.926569 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6cfd85-7fa9-41dc-b05e-81f1ede4f429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:48.927500 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6cfd85-7fa9-41dc-b05e-81f1ede4f429 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:48.928005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6cfd85-7fa9-41dc-b05e-81f1ede4f429 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:48.928620 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6cfd85-7fa9-41dc-b05e-81f1ede4f429 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:48.929766 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6cfd85-7fa9-41dc-b05e-81f1ede4f429 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:48.932309 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6cfd85-7fa9-41dc-b05e-81f1ede4f429 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:48.971795 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:49.083440 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 924/1871] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:26:48 2022] POST /identity/v3/projects => generated 373 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:49.089673 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638336b0-bacf-4c29-882e-de57a5be1c91 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:49.115384 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638336b0-bacf-4c29-882e-de57a5be1c91 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:49.210213 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a1f497-e6ff-45b2-b496-716f43ec51c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:49.210213 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1f497-e6ff-45b2-b496-716f43ec51c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:49.210213 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1f497-e6ff-45b2-b496-716f43ec51c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:49.210213 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1f497-e6ff-45b2-b496-716f43ec51c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:49.210612 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a1f497-e6ff-45b2-b496-716f43ec51c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:49.211682 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a1f497-e6ff-45b2-b496-716f43ec51c4 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:49.366525 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 948/1872] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:26:49 2022] POST /identity/v3/users => generated 287 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:49.371126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6cfd85-7fa9-41dc-b05e-81f1ede4f429 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:49.383391 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6cfd85-7fa9-41dc-b05e-81f1ede4f429 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:49.388966 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd932be7-12b0-478b-a033-58a7311a749a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:49.389789 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd932be7-12b0-478b-a033-58a7311a749a admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:49.390073 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd932be7-12b0-478b-a033-58a7311a749a admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:49.390355 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd932be7-12b0-478b-a033-58a7311a749a admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:49.391138 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd932be7-12b0-478b-a033-58a7311a749a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:49.392940 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd932be7-12b0-478b-a033-58a7311a749a admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:49.405785 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 925/1873] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 06:26:49.410462 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a1f497-e6ff-45b2-b496-716f43ec51c4 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:49.430221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a1f497-e6ff-45b2-b496-716f43ec51c4 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:49.435878 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31ad78d-ca01-41fa-9418-7771acb54e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:49.436540 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ad78d-ca01-41fa-9418-7771acb54e50 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:49.436840 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ad78d-ca01-41fa-9418-7771acb54e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:49.437275 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ad78d-ca01-41fa-9418-7771acb54e50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:49.437898 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31ad78d-ca01-41fa-9418-7771acb54e50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:49.439826 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31ad78d-ca01-41fa-9418-7771acb54e50 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:49.455432 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 949/1874] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:49.460098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd932be7-12b0-478b-a033-58a7311a749a admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:49.481437 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd932be7-12b0-478b-a033-58a7311a749a admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:49.488860 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a820a-eff7-45c8-a814-ba7d79d31e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:49.489955 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a820a-eff7-45c8-a814-ba7d79d31e12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:49.490306 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a820a-eff7-45c8-a814-ba7d79d31e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:49.490593 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a820a-eff7-45c8-a814-ba7d79d31e12 admin admin] PATH_INFO: `/v3/projects/bf300d2ae50647d993d8001f4e14c5ca/users/d133618cbb744e039a9e4dcee73aa1ac/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:49.519561 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8a820a-eff7-45c8-a814-ba7d79d31e12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf300d2ae50647d993d8001f4e14c5ca, user_id=d133618cbb744e039a9e4dcee73aa1ac, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:49.522718 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8a820a-eff7-45c8-a814-ba7d79d31e12 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:49.623862 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 926/1875] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:49 2022] PUT /identity/v3/projects/bf300d2ae50647d993d8001f4e14c5ca/users/d133618cbb744e039a9e4dcee73aa1ac/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:49.640239 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac802025-ca85-4df0-b993-c403e1387420 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:49.640568 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac802025-ca85-4df0-b993-c403e1387420 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:49.641437 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac802025-ca85-4df0-b993-c403e1387420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:49.701994 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-ac802025-ca85-4df0-b993-c403e1387420 None None] MFA Rules not processed for user `d133618cbb744e039a9e4dcee73aa1ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:49.704710 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ac802025-ca85-4df0-b993-c403e1387420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:49.847760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 950/1876] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:49.860505 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a820a-eff7-45c8-a814-ba7d79d31e12 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:49.902621 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:49.934243 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a820a-eff7-45c8-a814-ba7d79d31e12 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:49.940225 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8217109-043b-492f-b71c-d1259d437327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:49.941678 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8217109-043b-492f-b71c-d1259d437327 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:49.942684 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8217109-043b-492f-b71c-d1259d437327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:49.943499 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8217109-043b-492f-b71c-d1259d437327 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:49.947122 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:49.950647 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8217109-043b-492f-b71c-d1259d437327 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:49.950647 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8217109-043b-492f-b71c-d1259d437327 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:50.083211 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 927/1877] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:26:49 2022] POST /identity/v3/projects => generated 373 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:26:50.087449 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac802025-ca85-4df0-b993-c403e1387420 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:50.099549 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac802025-ca85-4df0-b993-c403e1387420 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:50.177055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a132e0-9721-4bdb-83b1-6cc078c3f51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:50.177810 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a132e0-9721-4bdb-83b1-6cc078c3f51c admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:50.178102 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a132e0-9721-4bdb-83b1-6cc078c3f51c admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:50.178384 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a132e0-9721-4bdb-83b1-6cc078c3f51c admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:50.179108 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a132e0-9721-4bdb-83b1-6cc078c3f51c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:50.181267 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a132e0-9721-4bdb-83b1-6cc078c3f51c admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:50.352978 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 951/1878] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:26:50 2022] POST /identity/v3/users => generated 293 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:50.357969 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8217109-043b-492f-b71c-d1259d437327 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:50.381043 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8217109-043b-492f-b71c-d1259d437327 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:50.389294 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c590345-a291-4062-a7e0-33d59dba18f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:50.390465 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c590345-a291-4062-a7e0-33d59dba18f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:50.390990 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c590345-a291-4062-a7e0-33d59dba18f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:50.391450 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c590345-a291-4062-a7e0-33d59dba18f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:50.392909 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c590345-a291-4062-a7e0-33d59dba18f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:50.397203 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c590345-a291-4062-a7e0-33d59dba18f0 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:50.420635 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 928/1879] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:50.427498 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a132e0-9721-4bdb-83b1-6cc078c3f51c admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:50.441472 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a132e0-9721-4bdb-83b1-6cc078c3f51c admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:50.449846 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a40a34-fb9b-438f-9905-e1ef2d78ad03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:50.451472 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a40a34-fb9b-438f-9905-e1ef2d78ad03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:50.451971 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a40a34-fb9b-438f-9905-e1ef2d78ad03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:50.452516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a40a34-fb9b-438f-9905-e1ef2d78ad03 admin admin] PATH_INFO: `/v3/projects/2a75bc0d4eee48e4b4edb576c5157d74/users/ed2709015209405fb025f190365160dc/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:50.493085 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a40a34-fb9b-438f-9905-e1ef2d78ad03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a75bc0d4eee48e4b4edb576c5157d74, user_id=ed2709015209405fb025f190365160dc, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:50.498773 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a40a34-fb9b-438f-9905-e1ef2d78ad03 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:50.662003 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 952/1880] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:50 2022] PUT /identity/v3/projects/2a75bc0d4eee48e4b4edb576c5157d74/users/ed2709015209405fb025f190365160dc/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:26:50.665789 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c590345-a291-4062-a7e0-33d59dba18f0 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:50.677749 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c590345-a291-4062-a7e0-33d59dba18f0 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:50.747346 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d16c65-0204-4a26-a605-287b29872548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:50.748079 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d16c65-0204-4a26-a605-287b29872548 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:50.748371 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d16c65-0204-4a26-a605-287b29872548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:50.748688 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d16c65-0204-4a26-a605-287b29872548 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:50.749549 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d16c65-0204-4a26-a605-287b29872548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:50.751300 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d16c65-0204-4a26-a605-287b29872548 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:50.763885 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 929/1881] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:50.769636 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a40a34-fb9b-438f-9905-e1ef2d78ad03 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:50.781101 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a40a34-fb9b-438f-9905-e1ef2d78ad03 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:50.786575 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27df033-74c9-4182-aa8f-5c5d358012a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:50.787816 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27df033-74c9-4182-aa8f-5c5d358012a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:50.788112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27df033-74c9-4182-aa8f-5c5d358012a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:50.788791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27df033-74c9-4182-aa8f-5c5d358012a6 admin admin] PATH_INFO: `/v3/projects/2a75bc0d4eee48e4b4edb576c5157d74/users/ed2709015209405fb025f190365160dc/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:50.792112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27df033-74c9-4182-aa8f-5c5d358012a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a75bc0d4eee48e4b4edb576c5157d74, user_id=ed2709015209405fb025f190365160dc, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:50.794563 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27df033-74c9-4182-aa8f-5c5d358012a6 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:50.806967 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 953/1882] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:50 2022] PUT /identity/v3/projects/2a75bc0d4eee48e4b4edb576c5157d74/users/ed2709015209405fb025f190365160dc/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:50.824610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b80554-d3db-4cb0-bc03-6da152555c9e None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:50.824908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b80554-d3db-4cb0-bc03-6da152555c9e None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:50.825194 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b80554-d3db-4cb0-bc03-6da152555c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:50.856219 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-f6b80554-d3db-4cb0-bc03-6da152555c9e None None] MFA Rules not processed for user `ed2709015209405fb025f190365160dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:50.857383 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f6b80554-d3db-4cb0-bc03-6da152555c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:50.865306 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:50.908663 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:50.990067 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 930/1883] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:50 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:51.003428 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27df033-74c9-4182-aa8f-5c5d358012a6 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:51.005607 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-b27df033-74c9-4182-aa8f-5c5d358012a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:51.086684 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27df033-74c9-4182-aa8f-5c5d358012a6 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:51.090740 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bec956-40b0-45aa-a8dd-298e0882be29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:51.091485 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bec956-40b0-45aa-a8dd-298e0882be29 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:51.091776 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bec956-40b0-45aa-a8dd-298e0882be29 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:51.092057 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bec956-40b0-45aa-a8dd-298e0882be29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:51.102088 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:51.103250 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bec956-40b0-45aa-a8dd-298e0882be29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:51.104853 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bec956-40b0-45aa-a8dd-298e0882be29 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:51.115652 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 954/1884] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:51 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:52.194027 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b80554-d3db-4cb0-bc03-6da152555c9e None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:52.271740 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b80554-d3db-4cb0-bc03-6da152555c9e None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:52.276923 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f8f3b-c0b5-4c7e-9b77-b48852cdca82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:52.277828 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f8f3b-c0b5-4c7e-9b77-b48852cdca82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:52.278113 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f8f3b-c0b5-4c7e-9b77-b48852cdca82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:52.278396 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f8f3b-c0b5-4c7e-9b77-b48852cdca82 admin admin] PATH_INFO: `/v3/users/d133618cbb744e039a9e4dcee73aa1ac` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:52.279683 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0f8f3b-c0b5-4c7e-9b77-b48852cdca82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d133618cbb744e039a9e4dcee73aa1ac)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:52.281752 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0f8f3b-c0b5-4c7e-9b77-b48852cdca82 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:52.462649 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-7d0f8f3b-c0b5-4c7e-9b77-b48852cdca82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd133618cbb744e039a9e4dcee73aa1ac'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:52.469788 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-7d0f8f3b-c0b5-4c7e-9b77-b48852cdca82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd133618cbb744e039a9e4dcee73aa1ac'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:52.484095 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-7d0f8f3b-c0b5-4c7e-9b77-b48852cdca82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd133618cbb744e039a9e4dcee73aa1ac'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:52.484889 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-7d0f8f3b-c0b5-4c7e-9b77-b48852cdca82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd133618cbb744e039a9e4dcee73aa1ac'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:52.485529 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-7d0f8f3b-c0b5-4c7e-9b77-b48852cdca82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd133618cbb744e039a9e4dcee73aa1ac'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:52.593329 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 931/1885] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:26:52 2022] DELETE /identity/v3/users/d133618cbb744e039a9e4dcee73aa1ac => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:52.599605 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a14fc4-edbf-47ae-9a36-c36479a28e1e None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:52.600137 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a14fc4-edbf-47ae-9a36-c36479a28e1e None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:52.600226 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a14fc4-edbf-47ae-9a36-c36479a28e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:52.619920 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-77a14fc4-edbf-47ae-9a36-c36479a28e1e None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:52.621462 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-77a14fc4-edbf-47ae-9a36-c36479a28e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:52.694350 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 955/1886] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:26:52.705206 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f8f3b-c0b5-4c7e-9b77-b48852cdca82 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:52.707950 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7d0f8f3b-c0b5-4c7e-9b77-b48852cdca82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:52.759799 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:52.787112 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f8f3b-c0b5-4c7e-9b77-b48852cdca82 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:52.793411 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e723d8a-bc55-4f61-9231-d4ea89633c0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:52.794568 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e723d8a-bc55-4f61-9231-d4ea89633c0b service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:52.795069 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e723d8a-bc55-4f61-9231-d4ea89633c0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:52.795569 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e723d8a-bc55-4f61-9231-d4ea89633c0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:52.813772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e723d8a-bc55-4f61-9231-d4ea89633c0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:52.817123 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e723d8a-bc55-4f61-9231-d4ea89633c0b service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:52.836838 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 932/1887] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:26:53.458122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a14fc4-edbf-47ae-9a36-c36479a28e1e None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:53.459730 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-77a14fc4-edbf-47ae-9a36-c36479a28e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:53.481266 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a14fc4-edbf-47ae-9a36-c36479a28e1e None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:53.485990 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9586c5-8f31-49eb-9d57-63caaddf5292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:53.496584 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9586c5-8f31-49eb-9d57-63caaddf5292 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:53.496999 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9586c5-8f31-49eb-9d57-63caaddf5292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:53.497303 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9586c5-8f31-49eb-9d57-63caaddf5292 admin admin] PATH_INFO: `/v3/projects/bf300d2ae50647d993d8001f4e14c5ca` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:53.498806 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9586c5-8f31-49eb-9d57-63caaddf5292 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf300d2ae50647d993d8001f4e14c5ca)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:53.500823 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9586c5-8f31-49eb-9d57-63caaddf5292 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:53.668759 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1f9586c5-8f31-49eb-9d57-63caaddf5292 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf300d2ae50647d993d8001f4e14c5ca', 'request_id': 'req-1f9586c5-8f31-49eb-9d57-63caaddf5292'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:53.676474 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1f9586c5-8f31-49eb-9d57-63caaddf5292 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf300d2ae50647d993d8001f4e14c5ca', 'request_id': 'req-1f9586c5-8f31-49eb-9d57-63caaddf5292'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:53.770701 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1f9586c5-8f31-49eb-9d57-63caaddf5292 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf300d2ae50647d993d8001f4e14c5ca', 'request_id': 'req-1f9586c5-8f31-49eb-9d57-63caaddf5292'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:53.785895 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1f9586c5-8f31-49eb-9d57-63caaddf5292 admin admin] The token cache is being invalidate because project bf300d2ae50647d993d8001f4e14c5ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:26:53.786204 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1f9586c5-8f31-49eb-9d57-63caaddf5292 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:53.787158 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1f9586c5-8f31-49eb-9d57-63caaddf5292 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:53.793489 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 956/1888] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:26:53 2022] DELETE /identity/v3/projects/bf300d2ae50647d993d8001f4e14c5ca => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:53.799410 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e723d8a-bc55-4f61-9231-d4ea89633c0b service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:53.821374 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e723d8a-bc55-4f61-9231-d4ea89633c0b service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:53.948089 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a3a76-2888-4e22-8377-5b2393a785b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:53.949061 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a3a76-2888-4e22-8377-5b2393a785b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:53.949720 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a3a76-2888-4e22-8377-5b2393a785b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:53.950192 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a3a76-2888-4e22-8377-5b2393a785b9 admin admin] PATH_INFO: `/v3/users/ed2709015209405fb025f190365160dc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:53.952375 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166a3a76-2888-4e22-8377-5b2393a785b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed2709015209405fb025f190365160dc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:53.955113 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166a3a76-2888-4e22-8377-5b2393a785b9 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:54.200111 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-166a3a76-2888-4e22-8377-5b2393a785b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed2709015209405fb025f190365160dc'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:54.224906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-166a3a76-2888-4e22-8377-5b2393a785b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed2709015209405fb025f190365160dc'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:54.240197 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-166a3a76-2888-4e22-8377-5b2393a785b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed2709015209405fb025f190365160dc'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:54.243715 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-166a3a76-2888-4e22-8377-5b2393a785b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed2709015209405fb025f190365160dc'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:54.244138 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-166a3a76-2888-4e22-8377-5b2393a785b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed2709015209405fb025f190365160dc'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:54.328540 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 933/1889] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:26:53 2022] DELETE /identity/v3/users/ed2709015209405fb025f190365160dc => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:55.119712 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9586c5-8f31-49eb-9d57-63caaddf5292 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:55.122567 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-1f9586c5-8f31-49eb-9d57-63caaddf5292 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:55.173535 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9586c5-8f31-49eb-9d57-63caaddf5292 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:55.210608 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:55.271370 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50dd32-ed1d-4398-83ce-7d78009594dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:55.272400 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50dd32-ed1d-4398-83ce-7d78009594dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:55.272722 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50dd32-ed1d-4398-83ce-7d78009594dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:55.273005 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50dd32-ed1d-4398-83ce-7d78009594dd admin admin] PATH_INFO: `/v3/projects/2a75bc0d4eee48e4b4edb576c5157d74` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:55.274283 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a50dd32-ed1d-4398-83ce-7d78009594dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a75bc0d4eee48e4b4edb576c5157d74)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:55.276238 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a50dd32-ed1d-4398-83ce-7d78009594dd admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:55.387000 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0a50dd32-ed1d-4398-83ce-7d78009594dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a75bc0d4eee48e4b4edb576c5157d74', 'request_id': 'req-0a50dd32-ed1d-4398-83ce-7d78009594dd'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:55.395571 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0a50dd32-ed1d-4398-83ce-7d78009594dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a75bc0d4eee48e4b4edb576c5157d74', 'request_id': 'req-0a50dd32-ed1d-4398-83ce-7d78009594dd'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:55.501339 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0a50dd32-ed1d-4398-83ce-7d78009594dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a75bc0d4eee48e4b4edb576c5157d74', 'request_id': 'req-0a50dd32-ed1d-4398-83ce-7d78009594dd'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:55.512915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0a50dd32-ed1d-4398-83ce-7d78009594dd admin admin] The token cache is being invalidate because project 2a75bc0d4eee48e4b4edb576c5157d74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:26:55.513246 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0a50dd32-ed1d-4398-83ce-7d78009594dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:55.513909 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0a50dd32-ed1d-4398-83ce-7d78009594dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:26:55.515500 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 957/1890] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:26:55 2022] DELETE /identity/v3/projects/2a75bc0d4eee48e4b4edb576c5157d74 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:26:55.522568 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ced14-a696-49fb-b330-90cf55e75b13 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:55.523287 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ced14-a696-49fb-b330-90cf55e75b13 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:55.523287 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ced14-a696-49fb-b330-90cf55e75b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:55.543243 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-6a9ced14-a696-49fb-b330-90cf55e75b13 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:55.544587 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-6a9ced14-a696-49fb-b330-90cf55e75b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:55.611227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:55.621594 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 934/1891] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:55.629724 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02405a2f-3034-4846-8dfe-426daafa575a None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:55.629912 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02405a2f-3034-4846-8dfe-426daafa575a None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:55.630192 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02405a2f-3034-4846-8dfe-426daafa575a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:55.652023 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-02405a2f-3034-4846-8dfe-426daafa575a None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:55.652023 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-02405a2f-3034-4846-8dfe-426daafa575a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:55.673134 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 958/1892] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:55.678487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9ced14-a696-49fb-b330-90cf55e75b13 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:55.703370 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9ced14-a696-49fb-b330-90cf55e75b13 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:55.711709 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838206a-76e1-4b0f-aab7-5240db31428e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:55.711709 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838206a-76e1-4b0f-aab7-5240db31428e admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:55.711709 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838206a-76e1-4b0f-aab7-5240db31428e admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:55.712150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838206a-76e1-4b0f-aab7-5240db31428e admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:55.712710 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0838206a-76e1-4b0f-aab7-5240db31428e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:26:55.713024 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0838206a-76e1-4b0f-aab7-5240db31428e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:55.714883 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0838206a-76e1-4b0f-aab7-5240db31428e admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:55.732739 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 935/1893] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:26:55.735003 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02405a2f-3034-4846-8dfe-426daafa575a None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:55.749733 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02405a2f-3034-4846-8dfe-426daafa575a None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:55.760866 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b70852-f8c8-42e9-af66-a9f3e4c62ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:55.761955 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b70852-f8c8-42e9-af66-a9f3e4c62ffa admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:55.763024 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b70852-f8c8-42e9-af66-a9f3e4c62ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:55.763024 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b70852-f8c8-42e9-af66-a9f3e4c62ffa admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:55.764665 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b70852-f8c8-42e9-af66-a9f3e4c62ffa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:55.768110 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b70852-f8c8-42e9-af66-a9f3e4c62ffa admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:55.932798 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 959/1894] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:26:55 2022] POST /identity/v3/projects => generated 385 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:26:55.937558 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838206a-76e1-4b0f-aab7-5240db31428e admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:55.948778 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838206a-76e1-4b0f-aab7-5240db31428e admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:56.037190 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019b6f7-c861-428e-a599-be238541a40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:56.038431 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019b6f7-c861-428e-a599-be238541a40b admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:56.039037 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019b6f7-c861-428e-a599-be238541a40b admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:56.040586 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019b6f7-c861-428e-a599-be238541a40b admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:56.042662 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2019b6f7-c861-428e-a599-be238541a40b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:56.047391 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2019b6f7-c861-428e-a599-be238541a40b admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:56.224465 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 936/1895] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:26:55 2022] POST /identity/v3/users => generated 293 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:26:56.227506 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b70852-f8c8-42e9-af66-a9f3e4c62ffa admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:56.243727 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b70852-f8c8-42e9-af66-a9f3e4c62ffa admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:56.268167 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea174b-ea3e-48f3-8c04-603930bcee56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:56.268889 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea174b-ea3e-48f3-8c04-603930bcee56 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:56.269177 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea174b-ea3e-48f3-8c04-603930bcee56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:56.269508 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea174b-ea3e-48f3-8c04-603930bcee56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:56.270260 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ea174b-ea3e-48f3-8c04-603930bcee56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:56.271942 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ea174b-ea3e-48f3-8c04-603930bcee56 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:56.283249 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 960/1896] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 06:26:56.287434 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019b6f7-c861-428e-a599-be238541a40b admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:56.297558 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019b6f7-c861-428e-a599-be238541a40b admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:56.303553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20c31b-f4b3-41cd-be32-e275432649c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:56.304343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20c31b-f4b3-41cd-be32-e275432649c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:56.304758 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20c31b-f4b3-41cd-be32-e275432649c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:56.305028 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20c31b-f4b3-41cd-be32-e275432649c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:56.305905 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f20c31b-f4b3-41cd-be32-e275432649c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:56.308664 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f20c31b-f4b3-41cd-be32-e275432649c7 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:56.323070 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 937/1897] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:26:56.327496 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea174b-ea3e-48f3-8c04-603930bcee56 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:56.338845 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea174b-ea3e-48f3-8c04-603930bcee56 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:56.344339 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817f0b80-dc7e-40ca-9260-61178dc91256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:56.345397 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f0b80-dc7e-40ca-9260-61178dc91256 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:56.345750 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f0b80-dc7e-40ca-9260-61178dc91256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:56.346100 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f0b80-dc7e-40ca-9260-61178dc91256 admin admin] PATH_INFO: `/v3/projects/a5fcebce62564cab9e0a4450d93670ee/users/9797166c6ef84691a191d88dc065fff6/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:56.364573 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817f0b80-dc7e-40ca-9260-61178dc91256 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5fcebce62564cab9e0a4450d93670ee, user_id=9797166c6ef84691a191d88dc065fff6, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:56.367281 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817f0b80-dc7e-40ca-9260-61178dc91256 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:56.447060 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 961/1898] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:26:56 2022] PUT /identity/v3/projects/a5fcebce62564cab9e0a4450d93670ee/users/9797166c6ef84691a191d88dc065fff6/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:26:56.473500 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2052e6-5721-4bfa-9ff7-6ccba8641bb1 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:56.478547 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2052e6-5721-4bfa-9ff7-6ccba8641bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:56.478881 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2052e6-5721-4bfa-9ff7-6ccba8641bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:56.511020 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-bb2052e6-5721-4bfa-9ff7-6ccba8641bb1 None None] MFA Rules not processed for user `9797166c6ef84691a191d88dc065fff6`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:26:56.512261 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-bb2052e6-5721-4bfa-9ff7-6ccba8641bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:56.624380 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 938/1899] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:26:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:26:56.638322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817f0b80-dc7e-40ca-9260-61178dc91256 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:26:56.640902 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-817f0b80-dc7e-40ca-9260-61178dc91256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:26:56.744893 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:26:56.755392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817f0b80-dc7e-40ca-9260-61178dc91256 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:26:56.759174 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286b2fd-37ca-4f7a-91e9-62760fecf1b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:26:56.762334 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286b2fd-37ca-4f7a-91e9-62760fecf1b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:26:56.762334 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286b2fd-37ca-4f7a-91e9-62760fecf1b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:26:56.762334 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286b2fd-37ca-4f7a-91e9-62760fecf1b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:26:56.770704 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7286b2fd-37ca-4f7a-91e9-62760fecf1b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:26:56.772152 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7286b2fd-37ca-4f7a-91e9-62760fecf1b1 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:26:56.783981 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 962/1900] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:26:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:27:01.244033 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2052e6-5721-4bfa-9ff7-6ccba8641bb1 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:01.245815 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-bb2052e6-5721-4bfa-9ff7-6ccba8641bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:01.259229 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2052e6-5721-4bfa-9ff7-6ccba8641bb1 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:01.263278 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd414cd-af27-4139-9b90-5b80acbebb47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:01.263994 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd414cd-af27-4139-9b90-5b80acbebb47 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:01.264277 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd414cd-af27-4139-9b90-5b80acbebb47 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:01.264589 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd414cd-af27-4139-9b90-5b80acbebb47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:01.266630 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-0bd414cd-af27-4139-9b90-5b80acbebb47 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:01.351082 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:01.368140 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd414cd-af27-4139-9b90-5b80acbebb47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:01.369897 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd414cd-af27-4139-9b90-5b80acbebb47 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:01.383833 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 939/1901] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:27:01 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:27:02.363964 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286b2fd-37ca-4f7a-91e9-62760fecf1b1 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:02.365539 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-7286b2fd-37ca-4f7a-91e9-62760fecf1b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:02.559074 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286b2fd-37ca-4f7a-91e9-62760fecf1b1 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:02.580274 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db5cbfb-0399-427d-af70-660c94a99bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:02.580274 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5cbfb-0399-427d-af70-660c94a99bab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:02.580274 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5cbfb-0399-427d-af70-660c94a99bab admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:02.580274 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5cbfb-0399-427d-af70-660c94a99bab admin admin] PATH_INFO: `/v3/users/a92c16fc4ba049ec8aa08c2ef9af31c9` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:02.585657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db5cbfb-0399-427d-af70-660c94a99bab admin admin] RBAC: Authorizing `identity:delete_user(user_id=a92c16fc4ba049ec8aa08c2ef9af31c9)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:02.587639 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db5cbfb-0399-427d-af70-660c94a99bab admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:02.856229 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0db5cbfb-0399-427d-af70-660c94a99bab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a92c16fc4ba049ec8aa08c2ef9af31c9'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:02.872609 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0db5cbfb-0399-427d-af70-660c94a99bab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a92c16fc4ba049ec8aa08c2ef9af31c9'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:02.873767 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0db5cbfb-0399-427d-af70-660c94a99bab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a92c16fc4ba049ec8aa08c2ef9af31c9'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:02.875957 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0db5cbfb-0399-427d-af70-660c94a99bab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a92c16fc4ba049ec8aa08c2ef9af31c9'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:02.899287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0db5cbfb-0399-427d-af70-660c94a99bab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a92c16fc4ba049ec8aa08c2ef9af31c9'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:03.000266 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 963/1902] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:27:02 2022] DELETE /identity/v3/users/a92c16fc4ba049ec8aa08c2ef9af31c9 => generated 0 bytes in 637 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:27:03.007227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24c188-0cdd-44f6-b565-12c2cec9e4b9 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:03.007880 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24c188-0cdd-44f6-b565-12c2cec9e4b9 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:03.007989 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24c188-0cdd-44f6-b565-12c2cec9e4b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:03.026373 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-ad24c188-0cdd-44f6-b565-12c2cec9e4b9 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:03.027656 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ad24c188-0cdd-44f6-b565-12c2cec9e4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:03.103741 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 940/1903] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:27:03.115374 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db5cbfb-0399-427d-af70-660c94a99bab admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:03.121252 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-0db5cbfb-0399-427d-af70-660c94a99bab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:03.184779 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db5cbfb-0399-427d-af70-660c94a99bab admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:03.188623 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51d36d-ba16-4b14-80c2-97f1643d005d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:03.189459 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51d36d-ba16-4b14-80c2-97f1643d005d service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:03.189747 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51d36d-ba16-4b14-80c2-97f1643d005d service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:03.190071 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51d36d-ba16-4b14-80c2-97f1643d005d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:03.199974 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c51d36d-ba16-4b14-80c2-97f1643d005d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:03.201462 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c51d36d-ba16-4b14-80c2-97f1643d005d service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:03.211795 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 964/1904] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:27:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:27:03.666769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24c188-0cdd-44f6-b565-12c2cec9e4b9 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:03.668307 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ad24c188-0cdd-44f6-b565-12c2cec9e4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:03.683708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24c188-0cdd-44f6-b565-12c2cec9e4b9 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:03.691606 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283b47f-2f42-4617-8a48-aadc2b3049ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:03.692575 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283b47f-2f42-4617-8a48-aadc2b3049ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:03.693061 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283b47f-2f42-4617-8a48-aadc2b3049ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:03.693282 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283b47f-2f42-4617-8a48-aadc2b3049ee admin admin] PATH_INFO: `/v3/projects/83993250dab64203bcefa94e341f6630` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:03.694837 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e283b47f-2f42-4617-8a48-aadc2b3049ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=83993250dab64203bcefa94e341f6630)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:03.698243 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e283b47f-2f42-4617-8a48-aadc2b3049ee admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:03.824319 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e283b47f-2f42-4617-8a48-aadc2b3049ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83993250dab64203bcefa94e341f6630', 'request_id': 'req-e283b47f-2f42-4617-8a48-aadc2b3049ee'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:03.836840 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e283b47f-2f42-4617-8a48-aadc2b3049ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83993250dab64203bcefa94e341f6630', 'request_id': 'req-e283b47f-2f42-4617-8a48-aadc2b3049ee'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:03.924039 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e283b47f-2f42-4617-8a48-aadc2b3049ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83993250dab64203bcefa94e341f6630', 'request_id': 'req-e283b47f-2f42-4617-8a48-aadc2b3049ee'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:03.939291 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e283b47f-2f42-4617-8a48-aadc2b3049ee admin admin] The token cache is being invalidate because project 83993250dab64203bcefa94e341f6630 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:27:03.939715 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e283b47f-2f42-4617-8a48-aadc2b3049ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:03.941078 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e283b47f-2f42-4617-8a48-aadc2b3049ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:03.943277 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 941/1905] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:27:03 2022] DELETE /identity/v3/projects/83993250dab64203bcefa94e341f6630 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:27:03.948003 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51d36d-ba16-4b14-80c2-97f1643d005d service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:03.960963 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51d36d-ba16-4b14-80c2-97f1643d005d service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:03.973094 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:04.039709 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41091df-5eda-4d7e-be08-c8bbf9191e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:04.040667 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41091df-5eda-4d7e-be08-c8bbf9191e38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:04.041366 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41091df-5eda-4d7e-be08-c8bbf9191e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:04.041701 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41091df-5eda-4d7e-be08-c8bbf9191e38 admin admin] PATH_INFO: `/v3/users/454eacbc2c4341e79ab33f6926d2d035` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:04.042970 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41091df-5eda-4d7e-be08-c8bbf9191e38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=454eacbc2c4341e79ab33f6926d2d035)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:04.044990 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41091df-5eda-4d7e-be08-c8bbf9191e38 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:04.118060 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283b47f-2f42-4617-8a48-aadc2b3049ee admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:04.119541 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e283b47f-2f42-4617-8a48-aadc2b3049ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:04.134630 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283b47f-2f42-4617-8a48-aadc2b3049ee admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:04.143922 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9c9c3-2900-4cd7-97ab-973fe4f32d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:04.143922 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9c9c3-2900-4cd7-97ab-973fe4f32d98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:04.143922 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9c9c3-2900-4cd7-97ab-973fe4f32d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:04.143922 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9c9c3-2900-4cd7-97ab-973fe4f32d98 admin admin] PATH_INFO: `/v3/users/6eb414ca9a5448c89b52f56250548034` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:04.144334 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc9c9c3-2900-4cd7-97ab-973fe4f32d98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6eb414ca9a5448c89b52f56250548034)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:04.146063 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc9c9c3-2900-4cd7-97ab-973fe4f32d98 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:04.308178 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-a41091df-5eda-4d7e-be08-c8bbf9191e38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '454eacbc2c4341e79ab33f6926d2d035'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:04.316184 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-a41091df-5eda-4d7e-be08-c8bbf9191e38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '454eacbc2c4341e79ab33f6926d2d035'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:04.337558 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-a41091df-5eda-4d7e-be08-c8bbf9191e38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '454eacbc2c4341e79ab33f6926d2d035'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:04.337558 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-a41091df-5eda-4d7e-be08-c8bbf9191e38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '454eacbc2c4341e79ab33f6926d2d035'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:04.337558 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-a41091df-5eda-4d7e-be08-c8bbf9191e38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '454eacbc2c4341e79ab33f6926d2d035'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:05.333222 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 965/1906] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:27:03 2022] DELETE /identity/v3/users/454eacbc2c4341e79ab33f6926d2d035 => generated 0 bytes in 1384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:27:05.335175 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3dc9c9c3-2900-4cd7-97ab-973fe4f32d98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6eb414ca9a5448c89b52f56250548034'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:05.349655 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3dc9c9c3-2900-4cd7-97ab-973fe4f32d98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6eb414ca9a5448c89b52f56250548034'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:05.369649 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:05.377317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3dc9c9c3-2900-4cd7-97ab-973fe4f32d98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6eb414ca9a5448c89b52f56250548034'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:05.379164 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3dc9c9c3-2900-4cd7-97ab-973fe4f32d98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6eb414ca9a5448c89b52f56250548034'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:05.380160 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3dc9c9c3-2900-4cd7-97ab-973fe4f32d98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6eb414ca9a5448c89b52f56250548034'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:05.530374 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 942/1907] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:27:04 2022] DELETE /identity/v3/users/6eb414ca9a5448c89b52f56250548034 => generated 0 bytes in 1413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:27:05.535194 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150947c0-4765-4809-a379-73ae8d915bca None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:05.535487 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150947c0-4765-4809-a379-73ae8d915bca None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:05.535806 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150947c0-4765-4809-a379-73ae8d915bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:05.565591 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-150947c0-4765-4809-a379-73ae8d915bca None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:05.567006 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-150947c0-4765-4809-a379-73ae8d915bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:05.653855 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 966/1908] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:27:05.674087 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9c9c3-2900-4cd7-97ab-973fe4f32d98 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:05.677063 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-3dc9c9c3-2900-4cd7-97ab-973fe4f32d98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:05.746380 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9c9c3-2900-4cd7-97ab-973fe4f32d98 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:05.746380 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a74258-12ec-43e2-ad15-04f8db144f40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:05.746380 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a74258-12ec-43e2-ad15-04f8db144f40 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:05.746380 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a74258-12ec-43e2-ad15-04f8db144f40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:05.747143 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a74258-12ec-43e2-ad15-04f8db144f40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:05.757902 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a74258-12ec-43e2-ad15-04f8db144f40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:05.763644 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a74258-12ec-43e2-ad15-04f8db144f40 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:05.776319 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 943/1909] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:27:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:27:05.860468 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150947c0-4765-4809-a379-73ae8d915bca None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:05.862982 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-150947c0-4765-4809-a379-73ae8d915bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:05.878779 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150947c0-4765-4809-a379-73ae8d915bca None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:05.885617 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083af248-2f08-4a19-aad7-f3f8558aa405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:05.886504 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083af248-2f08-4a19-aad7-f3f8558aa405 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:05.886795 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083af248-2f08-4a19-aad7-f3f8558aa405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:05.887080 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083af248-2f08-4a19-aad7-f3f8558aa405 admin admin] PATH_INFO: `/v3/projects/cb2fdea479d44caea280e8a747e12ddc` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:05.888458 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083af248-2f08-4a19-aad7-f3f8558aa405 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb2fdea479d44caea280e8a747e12ddc)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:05.891453 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083af248-2f08-4a19-aad7-f3f8558aa405 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:05.994475 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:06.011484 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-083af248-2f08-4a19-aad7-f3f8558aa405 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb2fdea479d44caea280e8a747e12ddc', 'request_id': 'req-083af248-2f08-4a19-aad7-f3f8558aa405'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:06.018706 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-083af248-2f08-4a19-aad7-f3f8558aa405 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb2fdea479d44caea280e8a747e12ddc', 'request_id': 'req-083af248-2f08-4a19-aad7-f3f8558aa405'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:06.124183 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-083af248-2f08-4a19-aad7-f3f8558aa405 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb2fdea479d44caea280e8a747e12ddc', 'request_id': 'req-083af248-2f08-4a19-aad7-f3f8558aa405'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:06.140198 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-083af248-2f08-4a19-aad7-f3f8558aa405 admin admin] The token cache is being invalidate because project cb2fdea479d44caea280e8a747e12ddc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:27:06.140629 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-083af248-2f08-4a19-aad7-f3f8558aa405 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:06.141249 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-083af248-2f08-4a19-aad7-f3f8558aa405 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:06.142905 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 967/1910] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:27:05 2022] DELETE /identity/v3/projects/cb2fdea479d44caea280e8a747e12ddc => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:27:06.151437 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5dbc09-187f-4631-a331-dd196028fdd3 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:06.151956 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5dbc09-187f-4631-a331-dd196028fdd3 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:06.152500 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5dbc09-187f-4631-a331-dd196028fdd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:06.176281 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-fb5dbc09-187f-4631-a331-dd196028fdd3 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:06.177573 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-fb5dbc09-187f-4631-a331-dd196028fdd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:06.248837 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 944/1911] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:27:06.258254 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0a4e9-fda4-485a-9f8b-aadd45cceb62 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:06.258622 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0a4e9-fda4-485a-9f8b-aadd45cceb62 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:06.259028 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0a4e9-fda4-485a-9f8b-aadd45cceb62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:06.273941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5dbc09-187f-4631-a331-dd196028fdd3 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:06.275992 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-fb5dbc09-187f-4631-a331-dd196028fdd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:06.286659 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-5fb0a4e9-fda4-485a-9f8b-aadd45cceb62 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:06.290355 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-5fb0a4e9-fda4-485a-9f8b-aadd45cceb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:06.294117 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5dbc09-187f-4631-a331-dd196028fdd3 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:06.299344 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e4d3a6-0c25-4e23-8087-6bb959061ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:06.300509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4d3a6-0c25-4e23-8087-6bb959061ca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:06.301103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4d3a6-0c25-4e23-8087-6bb959061ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:06.301220 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4d3a6-0c25-4e23-8087-6bb959061ca5 admin admin] PATH_INFO: `/v3/projects/7ac94c43103f4046a476a88d8b7028c8` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:06.302407 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e4d3a6-0c25-4e23-8087-6bb959061ca5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ac94c43103f4046a476a88d8b7028c8)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:06.304364 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e4d3a6-0c25-4e23-8087-6bb959061ca5 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:06.306061 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 968/1912] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:27:06.318428 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb0a4e9-fda4-485a-9f8b-aadd45cceb62 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:06.328535 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb0a4e9-fda4-485a-9f8b-aadd45cceb62 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:06.334576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac772b-ad08-4c98-8996-da1c84515504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:06.335308 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac772b-ad08-4c98-8996-da1c84515504 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:06.335545 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac772b-ad08-4c98-8996-da1c84515504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:06.335822 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac772b-ad08-4c98-8996-da1c84515504 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:06.338130 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ac772b-ad08-4c98-8996-da1c84515504 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:27:06.338130 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ac772b-ad08-4c98-8996-da1c84515504 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:06.338551 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ac772b-ad08-4c98-8996-da1c84515504 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:06.352971 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 969/1913] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:27:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:27:06.357497 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac772b-ad08-4c98-8996-da1c84515504 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:06.368774 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac772b-ad08-4c98-8996-da1c84515504 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:06.373893 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af7406-b329-485e-9a44-9d410ce1c2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:06.374499 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af7406-b329-485e-9a44-9d410ce1c2ff admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:06.374886 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af7406-b329-485e-9a44-9d410ce1c2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:06.375110 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af7406-b329-485e-9a44-9d410ce1c2ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:06.375845 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63af7406-b329-485e-9a44-9d410ce1c2ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:06.378345 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63af7406-b329-485e-9a44-9d410ce1c2ff admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:06.436870 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-a2e4d3a6-0c25-4e23-8087-6bb959061ca5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ac94c43103f4046a476a88d8b7028c8', 'request_id': 'req-a2e4d3a6-0c25-4e23-8087-6bb959061ca5'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:06.455191 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-a2e4d3a6-0c25-4e23-8087-6bb959061ca5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ac94c43103f4046a476a88d8b7028c8', 'request_id': 'req-a2e4d3a6-0c25-4e23-8087-6bb959061ca5'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:06.512189 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 970/1914] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:27:06 2022] POST /identity/v3/projects => generated 377 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:27:06.513282 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af7406-b329-485e-9a44-9d410ce1c2ff admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:06.524165 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-a2e4d3a6-0c25-4e23-8087-6bb959061ca5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ac94c43103f4046a476a88d8b7028c8', 'request_id': 'req-a2e4d3a6-0c25-4e23-8087-6bb959061ca5'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:06.539402 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-a2e4d3a6-0c25-4e23-8087-6bb959061ca5 admin admin] The token cache is being invalidate because project 7ac94c43103f4046a476a88d8b7028c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:27:06.539884 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-a2e4d3a6-0c25-4e23-8087-6bb959061ca5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:06.540497 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-a2e4d3a6-0c25-4e23-8087-6bb959061ca5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:06.545212 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 945/1915] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:27:06 2022] DELETE /identity/v3/projects/7ac94c43103f4046a476a88d8b7028c8 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:27:06.546439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e4d3a6-0c25-4e23-8087-6bb959061ca5 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:06.547720 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a2e4d3a6-0c25-4e23-8087-6bb959061ca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:06.564213 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:06.592780 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af7406-b329-485e-9a44-9d410ce1c2ff admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:06.603291 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed59cb-9d5b-4db2-8ca4-f11f5ce14e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:06.604081 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed59cb-9d5b-4db2-8ca4-f11f5ce14e0d admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:06.604377 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed59cb-9d5b-4db2-8ca4-f11f5ce14e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:06.604687 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed59cb-9d5b-4db2-8ca4-f11f5ce14e0d admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:06.605402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daed59cb-9d5b-4db2-8ca4-f11f5ce14e0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:06.607410 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daed59cb-9d5b-4db2-8ca4-f11f5ce14e0d admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:06.640086 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e4d3a6-0c25-4e23-8087-6bb959061ca5 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:06.648861 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0490881f-b211-4674-a13a-b1d5812c5391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:06.652844 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490881f-b211-4674-a13a-b1d5812c5391 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:06.653062 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490881f-b211-4674-a13a-b1d5812c5391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:06.653419 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490881f-b211-4674-a13a-b1d5812c5391 admin admin] PATH_INFO: `/v3/users/53bce8abe1774413a86ba2411c4603fe` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:06.654846 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0490881f-b211-4674-a13a-b1d5812c5391 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53bce8abe1774413a86ba2411c4603fe)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:06.658246 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0490881f-b211-4674-a13a-b1d5812c5391 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:06.728768 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 971/1916] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:27:06 2022] POST /identity/v3/users => generated 289 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:27:06.733720 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed59cb-9d5b-4db2-8ca4-f11f5ce14e0d admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:06.733720 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-daed59cb-9d5b-4db2-8ca4-f11f5ce14e0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:06.747849 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed59cb-9d5b-4db2-8ca4-f11f5ce14e0d admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:06.753359 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c67ba0-cc43-45c7-906b-682f4d5416d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:06.754068 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c67ba0-cc43-45c7-906b-682f4d5416d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:06.754324 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c67ba0-cc43-45c7-906b-682f4d5416d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:06.754606 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c67ba0-cc43-45c7-906b-682f4d5416d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:06.755367 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c67ba0-cc43-45c7-906b-682f4d5416d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:06.757017 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c67ba0-cc43-45c7-906b-682f4d5416d5 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:06.771173 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 972/1917] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:27:06.779528 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c67ba0-cc43-45c7-906b-682f4d5416d5 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:06.790855 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c67ba0-cc43-45c7-906b-682f4d5416d5 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:06.797234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba1e4b-1974-4a06-9e68-62a56330c116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:06.798060 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba1e4b-1974-4a06-9e68-62a56330c116 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:06.798354 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba1e4b-1974-4a06-9e68-62a56330c116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:06.798643 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba1e4b-1974-4a06-9e68-62a56330c116 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:06.800187 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aba1e4b-1974-4a06-9e68-62a56330c116 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:06.801770 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aba1e4b-1974-4a06-9e68-62a56330c116 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:06.812808 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 973/1918] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:27:06.817917 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba1e4b-1974-4a06-9e68-62a56330c116 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:06.829459 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba1e4b-1974-4a06-9e68-62a56330c116 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:06.835318 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd42fee7-87fa-4685-b6ea-26d133cf796d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:06.836240 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42fee7-87fa-4685-b6ea-26d133cf796d admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:06.836573 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42fee7-87fa-4685-b6ea-26d133cf796d admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:06.836852 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42fee7-87fa-4685-b6ea-26d133cf796d admin admin] PATH_INFO: `/v3/projects/93d5b71aa4684e1c8b3b723a45d640df/users/832438487d43400d8806215e7a0ae457/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:06.853964 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd42fee7-87fa-4685-b6ea-26d133cf796d admin admin] RBAC: Authorizing `identity:create_grant(project_id=93d5b71aa4684e1c8b3b723a45d640df, user_id=832438487d43400d8806215e7a0ae457, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:06.856755 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd42fee7-87fa-4685-b6ea-26d133cf796d admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:06.888095 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0490881f-b211-4674-a13a-b1d5812c5391 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53bce8abe1774413a86ba2411c4603fe'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:06.896247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0490881f-b211-4674-a13a-b1d5812c5391 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53bce8abe1774413a86ba2411c4603fe'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:06.909938 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0490881f-b211-4674-a13a-b1d5812c5391 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53bce8abe1774413a86ba2411c4603fe'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:06.910828 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0490881f-b211-4674-a13a-b1d5812c5391 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53bce8abe1774413a86ba2411c4603fe'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:06.911482 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0490881f-b211-4674-a13a-b1d5812c5391 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53bce8abe1774413a86ba2411c4603fe'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:06.936134 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 974/1919] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:27:06 2022] PUT /identity/v3/projects/93d5b71aa4684e1c8b3b723a45d640df/users/832438487d43400d8806215e7a0ae457/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:27:06.948265 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c97dd-b349-4139-80dc-6d8e9ae7e79a None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:06.948594 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c97dd-b349-4139-80dc-6d8e9ae7e79a None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:06.948879 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c97dd-b349-4139-80dc-6d8e9ae7e79a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:06.977731 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-493c97dd-b349-4139-80dc-6d8e9ae7e79a None None] MFA Rules not processed for user `832438487d43400d8806215e7a0ae457`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:06.978962 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-493c97dd-b349-4139-80dc-6d8e9ae7e79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:06.983028 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:DELETE=6,keystone:SELECT=88,keystone:INSERT=6 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:07.026799 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 946/1920] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:27:06 2022] DELETE /identity/v3/users/53bce8abe1774413a86ba2411c4603fe => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:27:07.057012 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 975/1921] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:27:07.063347 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0490881f-b211-4674-a13a-b1d5812c5391 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:07.064982 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-0490881f-b211-4674-a13a-b1d5812c5391 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:07.143523 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0490881f-b211-4674-a13a-b1d5812c5391 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:07.148587 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b12d36-1808-4ae5-9fdf-723087a3c89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:07.149273 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b12d36-1808-4ae5-9fdf-723087a3c89d admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:07.149564 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b12d36-1808-4ae5-9fdf-723087a3c89d admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:07.149848 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b12d36-1808-4ae5-9fdf-723087a3c89d admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:07.150573 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b12d36-1808-4ae5-9fdf-723087a3c89d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:07.152635 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b12d36-1808-4ae5-9fdf-723087a3c89d admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:07.303230 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 947/1922] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:27:07 2022] POST /identity/v3/projects => generated 377 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:27:07.307612 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c97dd-b349-4139-80dc-6d8e9ae7e79a None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:07.395391 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c97dd-b349-4139-80dc-6d8e9ae7e79a None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:07.477364 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db4ae1-39e6-4393-ac01-0004ef08ea52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:07.478085 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db4ae1-39e6-4393-ac01-0004ef08ea52 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:07.478380 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db4ae1-39e6-4393-ac01-0004ef08ea52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:07.478688 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db4ae1-39e6-4393-ac01-0004ef08ea52 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:07.479407 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21db4ae1-39e6-4393-ac01-0004ef08ea52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:07.481447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21db4ae1-39e6-4393-ac01-0004ef08ea52 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:07.578229 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b12d36-1808-4ae5-9fdf-723087a3c89d admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:07.581326 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b0b12d36-1808-4ae5-9fdf-723087a3c89d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:07.582555 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 976/1923] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:27:07 2022] POST /identity/v3/users => generated 289 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:27:07.587212 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db4ae1-39e6-4393-ac01-0004ef08ea52 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:07.595426 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b12d36-1808-4ae5-9fdf-723087a3c89d admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:07.602364 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f425e2-b215-4c72-95d4-b5af062df5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:07.603336 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f425e2-b215-4c72-95d4-b5af062df5ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:07.603703 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f425e2-b215-4c72-95d4-b5af062df5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:07.604056 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f425e2-b215-4c72-95d4-b5af062df5ad admin admin] PATH_INFO: `/v3/projects/b998419218344ecaa336c7476cc8cbaf` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:07.604398 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db4ae1-39e6-4393-ac01-0004ef08ea52 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:07.605326 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f425e2-b215-4c72-95d4-b5af062df5ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=b998419218344ecaa336c7476cc8cbaf)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:07.607391 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f425e2-b215-4c72-95d4-b5af062df5ad admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:07.610738 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44906087-78c2-47b3-a74f-583aee34380e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:07.611704 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44906087-78c2-47b3-a74f-583aee34380e admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:07.612155 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44906087-78c2-47b3-a74f-583aee34380e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:07.612505 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44906087-78c2-47b3-a74f-583aee34380e admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:07.613204 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44906087-78c2-47b3-a74f-583aee34380e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:07.614849 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44906087-78c2-47b3-a74f-583aee34380e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:07.622472 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:07.630079 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 977/1924] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 06:27:07.633051 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44906087-78c2-47b3-a74f-583aee34380e admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:07.645088 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44906087-78c2-47b3-a74f-583aee34380e admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:07.650397 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7082dec2-dfdb-4cdb-a5f2-010adaddf849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:07.650995 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082dec2-dfdb-4cdb-a5f2-010adaddf849 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:07.651279 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082dec2-dfdb-4cdb-a5f2-010adaddf849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:07.651559 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082dec2-dfdb-4cdb-a5f2-010adaddf849 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:07.652298 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7082dec2-dfdb-4cdb-a5f2-010adaddf849 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:07.654013 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7082dec2-dfdb-4cdb-a5f2-010adaddf849 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:07.671382 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 978/1925] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:27:07.676093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7082dec2-dfdb-4cdb-a5f2-010adaddf849 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:07.699740 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7082dec2-dfdb-4cdb-a5f2-010adaddf849 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:07.715717 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a93144-ec6d-4710-99b6-b0983b21db94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:07.715717 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a93144-ec6d-4710-99b6-b0983b21db94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:07.715717 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a93144-ec6d-4710-99b6-b0983b21db94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:07.715717 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a93144-ec6d-4710-99b6-b0983b21db94 admin admin] PATH_INFO: `/v3/projects/f93d922736d04c9caefeaeedb77fbb44/users/7678ca998ea44013ba079a9f4daed196/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:07.743594 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a93144-ec6d-4710-99b6-b0983b21db94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f93d922736d04c9caefeaeedb77fbb44, user_id=7678ca998ea44013ba079a9f4daed196, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:07.747208 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a93144-ec6d-4710-99b6-b0983b21db94 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:07.751398 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:07.786530 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b7f425e2-b215-4c72-95d4-b5af062df5ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b998419218344ecaa336c7476cc8cbaf', 'request_id': 'req-b7f425e2-b215-4c72-95d4-b5af062df5ad'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:07.815967 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b7f425e2-b215-4c72-95d4-b5af062df5ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b998419218344ecaa336c7476cc8cbaf', 'request_id': 'req-b7f425e2-b215-4c72-95d4-b5af062df5ad'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:07.848184 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 979/1926] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:27:07 2022] PUT /identity/v3/projects/f93d922736d04c9caefeaeedb77fbb44/users/7678ca998ea44013ba079a9f4daed196/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:27:07.859647 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7ca2b-b93f-4f0b-869f-363b2fd37a71 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:07.860045 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7ca2b-b93f-4f0b-869f-363b2fd37a71 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:07.860338 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7ca2b-b93f-4f0b-869f-363b2fd37a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:07.892469 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-e8d7ca2b-b93f-4f0b-869f-363b2fd37a71 None None] MFA Rules not processed for user `7678ca998ea44013ba079a9f4daed196`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:07.894660 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e8d7ca2b-b93f-4f0b-869f-363b2fd37a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:07.931785 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b7f425e2-b215-4c72-95d4-b5af062df5ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b998419218344ecaa336c7476cc8cbaf', 'request_id': 'req-b7f425e2-b215-4c72-95d4-b5af062df5ad'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:07.951986 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b7f425e2-b215-4c72-95d4-b5af062df5ad admin admin] The token cache is being invalidate because project b998419218344ecaa336c7476cc8cbaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:27:07.952855 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b7f425e2-b215-4c72-95d4-b5af062df5ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:07.954094 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b7f425e2-b215-4c72-95d4-b5af062df5ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:07.961245 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 948/1927] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:27:07 2022] DELETE /identity/v3/projects/b998419218344ecaa336c7476cc8cbaf => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:27:07.969238 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079b7e3-77f2-4a38-9fbf-14351c29673c None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:07.969238 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079b7e3-77f2-4a38-9fbf-14351c29673c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:07.969238 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079b7e3-77f2-4a38-9fbf-14351c29673c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:08.004465 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-8079b7e3-77f2-4a38-9fbf-14351c29673c None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:08.005866 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8079b7e3-77f2-4a38-9fbf-14351c29673c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:08.014271 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 980/1928] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:27:08.023349 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d7ca2b-b93f-4f0b-869f-363b2fd37a71 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:08.023349 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e8d7ca2b-b93f-4f0b-869f-363b2fd37a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:08.108475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 949/1929] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:27:08.124073 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9185ba-11cb-490b-8c18-15da4e1b6c4b None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:08.124467 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9185ba-11cb-490b-8c18-15da4e1b6c4b None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:08.124761 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9185ba-11cb-490b-8c18-15da4e1b6c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:08.132523 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d7ca2b-b93f-4f0b-869f-363b2fd37a71 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:08.139747 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681195d1-ad86-4bca-a495-fb230434018b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:08.140616 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681195d1-ad86-4bca-a495-fb230434018b admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:08.141007 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681195d1-ad86-4bca-a495-fb230434018b admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:08.141369 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681195d1-ad86-4bca-a495-fb230434018b admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:08.142264 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681195d1-ad86-4bca-a495-fb230434018b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:08.145891 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681195d1-ad86-4bca-a495-fb230434018b admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:08.154435 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-4b9185ba-11cb-490b-8c18-15da4e1b6c4b None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:08.156578 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4b9185ba-11cb-490b-8c18-15da4e1b6c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:08.172947 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 950/1930] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:27:08.178100 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9185ba-11cb-490b-8c18-15da4e1b6c4b None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:08.188162 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9185ba-11cb-490b-8c18-15da4e1b6c4b None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:08.194045 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a41ff4-040d-4d24-8a21-442fb4e60e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:08.194725 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a41ff4-040d-4d24-8a21-442fb4e60e7d admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:08.195174 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a41ff4-040d-4d24-8a21-442fb4e60e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:08.195458 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a41ff4-040d-4d24-8a21-442fb4e60e7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:08.196205 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a41ff4-040d-4d24-8a21-442fb4e60e7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:27:08.196594 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a41ff4-040d-4d24-8a21-442fb4e60e7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:08.198201 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a41ff4-040d-4d24-8a21-442fb4e60e7d admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:08.211717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 951/1931] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:27:08.216660 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a41ff4-040d-4d24-8a21-442fb4e60e7d admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:08.234244 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a41ff4-040d-4d24-8a21-442fb4e60e7d admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:08.239328 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f988a964-5715-4d0b-a29a-5bc921f3ca97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:08.239983 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988a964-5715-4d0b-a29a-5bc921f3ca97 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:08.240270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988a964-5715-4d0b-a29a-5bc921f3ca97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:08.240589 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988a964-5715-4d0b-a29a-5bc921f3ca97 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:08.241348 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f988a964-5715-4d0b-a29a-5bc921f3ca97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:08.243280 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f988a964-5715-4d0b-a29a-5bc921f3ca97 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:08.257967 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 981/1932] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:27:08 2022] POST /identity/v3/projects => generated 375 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:27:08.263930 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681195d1-ad86-4bca-a495-fb230434018b admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:08.332963 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681195d1-ad86-4bca-a495-fb230434018b admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:08.337897 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f3fad-ff59-4019-b273-6906561f98a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:08.338703 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f3fad-ff59-4019-b273-6906561f98a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:08.339004 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f3fad-ff59-4019-b273-6906561f98a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:08.339289 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f3fad-ff59-4019-b273-6906561f98a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:08.340032 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5f3fad-ff59-4019-b273-6906561f98a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:08.342059 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5f3fad-ff59-4019-b273-6906561f98a2 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:08.361768 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 952/1933] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:27:08 2022] POST /identity/v3/projects => generated 379 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:27:08.366606 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f988a964-5715-4d0b-a29a-5bc921f3ca97 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:08.435067 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:08.439219 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f988a964-5715-4d0b-a29a-5bc921f3ca97 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:08.446408 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07f3b0-02e4-404b-b7d8-441105461969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:08.447582 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07f3b0-02e4-404b-b7d8-441105461969 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:08.448025 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07f3b0-02e4-404b-b7d8-441105461969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:08.448345 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07f3b0-02e4-404b-b7d8-441105461969 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:08.449112 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd07f3b0-02e4-404b-b7d8-441105461969 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:08.451199 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd07f3b0-02e4-404b-b7d8-441105461969 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:08.498223 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:08.506310 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 982/1934] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:27:08 2022] POST /identity/v3/users => generated 294 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:27:08.511728 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f3fad-ff59-4019-b273-6906561f98a2 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:08.528275 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f3fad-ff59-4019-b273-6906561f98a2 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:08.536133 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104d3e1-ae23-433d-af08-95e2928d185e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:08.537352 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104d3e1-ae23-433d-af08-95e2928d185e admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:08.537860 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104d3e1-ae23-433d-af08-95e2928d185e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:08.538347 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104d3e1-ae23-433d-af08-95e2928d185e admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:08.539777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104d3e1-ae23-433d-af08-95e2928d185e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:08.542886 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104d3e1-ae23-433d-af08-95e2928d185e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:08.557310 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 983/1935] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 06:27:08.561713 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104d3e1-ae23-433d-af08-95e2928d185e admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:08.577759 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104d3e1-ae23-433d-af08-95e2928d185e admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:08.586516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b26cbdc-7ef6-44b7-9708-b7b099b5144a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:08.587601 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26cbdc-7ef6-44b7-9708-b7b099b5144a admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:08.587986 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26cbdc-7ef6-44b7-9708-b7b099b5144a admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:08.588372 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26cbdc-7ef6-44b7-9708-b7b099b5144a admin admin] PATH_INFO: `/v3/projects/f6478d3d7bf04fd988ef5d7595108080/users/b8984365b1db4f06bf0c346e865e4796/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:08.605724 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b26cbdc-7ef6-44b7-9708-b7b099b5144a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6478d3d7bf04fd988ef5d7595108080, user_id=b8984365b1db4f06bf0c346e865e4796, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:08.608560 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b26cbdc-7ef6-44b7-9708-b7b099b5144a admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:08.629008 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 953/1936] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:27:08 2022] POST /identity/v3/users => generated 290 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:27:08.634211 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07f3b0-02e4-404b-b7d8-441105461969 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:08.645914 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07f3b0-02e4-404b-b7d8-441105461969 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:08.651226 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c95df-cb71-44bb-8c2d-9eec7544f345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:08.651933 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c95df-cb71-44bb-8c2d-9eec7544f345 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:08.652221 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c95df-cb71-44bb-8c2d-9eec7544f345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:08.652672 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c95df-cb71-44bb-8c2d-9eec7544f345 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:08.653445 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c95df-cb71-44bb-8c2d-9eec7544f345 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:08.655165 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c95df-cb71-44bb-8c2d-9eec7544f345 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:08.667275 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 954/1937] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 06:27:08.671906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c95df-cb71-44bb-8c2d-9eec7544f345 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:08.682631 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c95df-cb71-44bb-8c2d-9eec7544f345 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:08.687826 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac813f7c-ec4d-4349-97f2-cc679cadb130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:08.688400 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac813f7c-ec4d-4349-97f2-cc679cadb130 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:08.688713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac813f7c-ec4d-4349-97f2-cc679cadb130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:08.688999 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac813f7c-ec4d-4349-97f2-cc679cadb130 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:08.689737 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac813f7c-ec4d-4349-97f2-cc679cadb130 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:08.691826 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac813f7c-ec4d-4349-97f2-cc679cadb130 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:08.702892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 955/1938] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 06:27:08.708008 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac813f7c-ec4d-4349-97f2-cc679cadb130 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:08.717918 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac813f7c-ec4d-4349-97f2-cc679cadb130 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:08.728900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed073352-4324-4033-8338-b72aae816fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:08.728900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed073352-4324-4033-8338-b72aae816fe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:08.728900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed073352-4324-4033-8338-b72aae816fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:08.728900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed073352-4324-4033-8338-b72aae816fe8 admin admin] PATH_INFO: `/v3/projects/5734791ff2694a5eafc455e79878520a/users/8b5a3e1b06d34993babc9a95d5a3f1ab/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:08.741632 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed073352-4324-4033-8338-b72aae816fe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5734791ff2694a5eafc455e79878520a, user_id=8b5a3e1b06d34993babc9a95d5a3f1ab, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:08.744324 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed073352-4324-4033-8338-b72aae816fe8 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:08.758398 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 984/1939] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:27:08 2022] PUT /identity/v3/projects/f6478d3d7bf04fd988ef5d7595108080/users/b8984365b1db4f06bf0c346e865e4796/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:27:08.762347 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b26cbdc-7ef6-44b7-9708-b7b099b5144a admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:08.822018 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 956/1940] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:27:08 2022] PUT /identity/v3/projects/5734791ff2694a5eafc455e79878520a/users/8b5a3e1b06d34993babc9a95d5a3f1ab/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:27:08.827577 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b26cbdc-7ef6-44b7-9708-b7b099b5144a admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:08.833265 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9740c3db-0cdd-480c-8961-967a7bd90239 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:08.835581 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9740c3db-0cdd-480c-8961-967a7bd90239 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:08.835581 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9740c3db-0cdd-480c-8961-967a7bd90239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:08.863578 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-9740c3db-0cdd-480c-8961-967a7bd90239 None None] MFA Rules not processed for user `8b5a3e1b06d34993babc9a95d5a3f1ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:08.864825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-9740c3db-0cdd-480c-8961-967a7bd90239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:08.890919 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f14ab-0f91-4440-909b-44a3364710d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:08.891711 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f14ab-0f91-4440-909b-44a3364710d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:08.891998 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f14ab-0f91-4440-909b-44a3364710d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:08.892280 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f14ab-0f91-4440-909b-44a3364710d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:08.893580 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4f14ab-0f91-4440-909b-44a3364710d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:08.895340 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4f14ab-0f91-4440-909b-44a3364710d2 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:08.905514 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 985/1941] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 06:27:08.910302 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f14ab-0f91-4440-909b-44a3364710d2 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:08.920273 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f14ab-0f91-4440-909b-44a3364710d2 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:08.925019 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c1d021-3187-4615-b3c5-774c08ce5b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:08.925891 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1d021-3187-4615-b3c5-774c08ce5b30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:08.926185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1d021-3187-4615-b3c5-774c08ce5b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:08.926469 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1d021-3187-4615-b3c5-774c08ce5b30 admin admin] PATH_INFO: `/v3/projects/f6478d3d7bf04fd988ef5d7595108080/users/b8984365b1db4f06bf0c346e865e4796/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:08.929523 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c1d021-3187-4615-b3c5-774c08ce5b30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6478d3d7bf04fd988ef5d7595108080, user_id=b8984365b1db4f06bf0c346e865e4796, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:08.931268 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c1d021-3187-4615-b3c5-774c08ce5b30 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:08.933612 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 957/1942] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:27:08.938247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9740c3db-0cdd-480c-8961-967a7bd90239 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:08.942706 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 986/1943] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:27:08 2022] PUT /identity/v3/projects/f6478d3d7bf04fd988ef5d7595108080/users/b8984365b1db4f06bf0c346e865e4796/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:27:08.949904 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9740c3db-0cdd-480c-8961-967a7bd90239 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:08.952865 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ae036-6d51-46a9-b66e-3dec5c48335c None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:08.953162 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ae036-6d51-46a9-b66e-3dec5c48335c None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:08.953452 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ae036-6d51-46a9-b66e-3dec5c48335c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:08.982549 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-153ae036-6d51-46a9-b66e-3dec5c48335c None None] MFA Rules not processed for user `b8984365b1db4f06bf0c346e865e4796`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:08.983783 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-153ae036-6d51-46a9-b66e-3dec5c48335c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:09.017620 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322bc08a-af3d-48c3-9ab8-610a2135d038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:09.019484 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322bc08a-af3d-48c3-9ab8-610a2135d038 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:09.019780 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322bc08a-af3d-48c3-9ab8-610a2135d038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:09.020064 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322bc08a-af3d-48c3-9ab8-610a2135d038 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:09.020807 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322bc08a-af3d-48c3-9ab8-610a2135d038 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:09.022854 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322bc08a-af3d-48c3-9ab8-610a2135d038 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:09.046021 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:09.076537 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 987/1944] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:08 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:27:09.090771 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153ae036-6d51-46a9-b66e-3dec5c48335c None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:09.092704 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-153ae036-6d51-46a9-b66e-3dec5c48335c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:09.168652 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153ae036-6d51-46a9-b66e-3dec5c48335c None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:09.177935 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d7502-7cd8-404a-832f-864840d9670d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:09.179139 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d7502-7cd8-404a-832f-864840d9670d service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:09.180610 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d7502-7cd8-404a-832f-864840d9670d service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:09.180998 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d7502-7cd8-404a-832f-864840d9670d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:09.183398 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-203d7502-7cd8-404a-832f-864840d9670d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:09.194199 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 958/1945] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:27:08 2022] POST /identity/v3/projects => generated 377 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:27:09.195554 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322bc08a-af3d-48c3-9ab8-610a2135d038 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:09.225366 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:09.281420 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203d7502-7cd8-404a-832f-864840d9670d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:09.282990 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203d7502-7cd8-404a-832f-864840d9670d service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:09.293900 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 988/1946] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:27:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 06:27:09.302453 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322bc08a-af3d-48c3-9ab8-610a2135d038 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:09.307386 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e797e08-91a6-4f3e-a4e6-62ce853638e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:09.308066 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e797e08-91a6-4f3e-a4e6-62ce853638e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:09.308352 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e797e08-91a6-4f3e-a4e6-62ce853638e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:09.308664 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e797e08-91a6-4f3e-a4e6-62ce853638e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:09.309369 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e797e08-91a6-4f3e-a4e6-62ce853638e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:09.311377 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e797e08-91a6-4f3e-a4e6-62ce853638e0 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:09.433416 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 959/1947] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:27:09 2022] POST /identity/v3/users => generated 289 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:27:09.434666 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d7502-7cd8-404a-832f-864840d9670d service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:09.446558 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d7502-7cd8-404a-832f-864840d9670d service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:09.451918 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338dd110-b22a-4940-9a09-fffa9e178f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:09.452648 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338dd110-b22a-4940-9a09-fffa9e178f58 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:09.452971 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338dd110-b22a-4940-9a09-fffa9e178f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:09.453276 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338dd110-b22a-4940-9a09-fffa9e178f58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:09.454044 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338dd110-b22a-4940-9a09-fffa9e178f58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:09.456327 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338dd110-b22a-4940-9a09-fffa9e178f58 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:09.467135 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 989/1948] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 06:27:09.471379 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e797e08-91a6-4f3e-a4e6-62ce853638e0 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:09.481925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e797e08-91a6-4f3e-a4e6-62ce853638e0 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:09.486851 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e02365-2ed0-407b-8dc8-a0e0b822326d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:09.487540 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e02365-2ed0-407b-8dc8-a0e0b822326d admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:09.487832 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e02365-2ed0-407b-8dc8-a0e0b822326d admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:09.488137 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e02365-2ed0-407b-8dc8-a0e0b822326d admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:09.488931 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e02365-2ed0-407b-8dc8-a0e0b822326d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:09.490537 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e02365-2ed0-407b-8dc8-a0e0b822326d admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:09.501708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 960/1949] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:27:09.506519 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338dd110-b22a-4940-9a09-fffa9e178f58 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:09.517244 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338dd110-b22a-4940-9a09-fffa9e178f58 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:09.522724 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c921a-d112-4527-bfcd-bb83c4e8f5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:09.523634 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c921a-d112-4527-bfcd-bb83c4e8f5a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:09.523925 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c921a-d112-4527-bfcd-bb83c4e8f5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:09.524203 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c921a-d112-4527-bfcd-bb83c4e8f5a1 admin admin] PATH_INFO: `/v3/projects/d544338ddcc147fa900a140eeeb5ff87/users/607d6f7410eb477e97136cf6d85bedbb/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:09.541634 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422c921a-d112-4527-bfcd-bb83c4e8f5a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d544338ddcc147fa900a140eeeb5ff87, user_id=607d6f7410eb477e97136cf6d85bedbb, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:09.544365 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422c921a-d112-4527-bfcd-bb83c4e8f5a1 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:09.670587 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 990/1950] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:27:09 2022] PUT /identity/v3/projects/d544338ddcc147fa900a140eeeb5ff87/users/607d6f7410eb477e97136cf6d85bedbb/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:27:09.683778 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66e7b2-ffdb-4887-9011-e0fa2e980bba None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:09.684315 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66e7b2-ffdb-4887-9011-e0fa2e980bba None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:09.684837 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66e7b2-ffdb-4887-9011-e0fa2e980bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:09.734962 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-3f66e7b2-ffdb-4887-9011-e0fa2e980bba None None] MFA Rules not processed for user `607d6f7410eb477e97136cf6d85bedbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:09.737554 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-3f66e7b2-ffdb-4887-9011-e0fa2e980bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:09.822320 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 961/1951] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:27:09.826990 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c921a-d112-4527-bfcd-bb83c4e8f5a1 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:09.903610 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c921a-d112-4527-bfcd-bb83c4e8f5a1 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:09.909143 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b11347-2d02-40b9-9715-7bf74bd23212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:09.909829 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b11347-2d02-40b9-9715-7bf74bd23212 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:09.910152 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b11347-2d02-40b9-9715-7bf74bd23212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:09.910438 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b11347-2d02-40b9-9715-7bf74bd23212 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:09.911177 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b11347-2d02-40b9-9715-7bf74bd23212 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:09.913228 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b11347-2d02-40b9-9715-7bf74bd23212 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:09.916776 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:10.047462 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 991/1952] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:27:09 2022] POST /identity/v3/projects => generated 377 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:27:10.051413 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f66e7b2-ffdb-4887-9011-e0fa2e980bba None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:10.064225 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f66e7b2-ffdb-4887-9011-e0fa2e980bba None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:10.132150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3b5fb-207c-4981-ba66-23d591dc57d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:10.132946 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3b5fb-207c-4981-ba66-23d591dc57d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:10.133349 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3b5fb-207c-4981-ba66-23d591dc57d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:10.133631 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3b5fb-207c-4981-ba66-23d591dc57d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:10.134527 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b3b5fb-207c-4981-ba66-23d591dc57d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:10.136750 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b3b5fb-207c-4981-ba66-23d591dc57d4 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:10.325268 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 962/1953] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:27:10 2022] POST /identity/v3/users => generated 295 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:27:10.333041 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b11347-2d02-40b9-9715-7bf74bd23212 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:10.341080 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b11347-2d02-40b9-9715-7bf74bd23212 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:10.346875 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cd3fb-4425-40c0-a85d-3aed41567b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:10.347588 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cd3fb-4425-40c0-a85d-3aed41567b33 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:10.347871 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cd3fb-4425-40c0-a85d-3aed41567b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:10.348154 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cd3fb-4425-40c0-a85d-3aed41567b33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:10.348939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3cd3fb-4425-40c0-a85d-3aed41567b33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:10.350637 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3cd3fb-4425-40c0-a85d-3aed41567b33 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:10.367625 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 992/1954] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:27:10.375158 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3b5fb-207c-4981-ba66-23d591dc57d4 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:10.389733 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3b5fb-207c-4981-ba66-23d591dc57d4 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:10.393371 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b7533-7684-433c-86f5-cbd0bf62be93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:10.394522 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b7533-7684-433c-86f5-cbd0bf62be93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:10.394914 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b7533-7684-433c-86f5-cbd0bf62be93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:10.395254 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b7533-7684-433c-86f5-cbd0bf62be93 admin admin] PATH_INFO: `/v3/projects/3b992135e3c54732b17ddc6aa0eeba29/users/f3b11a4660f249fcb6a11f4c6290e504/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:10.410442 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:10.414696 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33b7533-7684-433c-86f5-cbd0bf62be93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b992135e3c54732b17ddc6aa0eeba29, user_id=f3b11a4660f249fcb6a11f4c6290e504, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:10.417427 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33b7533-7684-433c-86f5-cbd0bf62be93 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:10.550427 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 963/1955] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:27:10 2022] PUT /identity/v3/projects/3b992135e3c54732b17ddc6aa0eeba29/users/f3b11a4660f249fcb6a11f4c6290e504/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:27:10.554690 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cd3fb-4425-40c0-a85d-3aed41567b33 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:10.565713 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cd3fb-4425-40c0-a85d-3aed41567b33 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:10.643692 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ab36f-800a-4864-92eb-7ef45420ccc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:10.646230 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ab36f-800a-4864-92eb-7ef45420ccc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:10.646230 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ab36f-800a-4864-92eb-7ef45420ccc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:10.646230 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ab36f-800a-4864-92eb-7ef45420ccc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:10.646230 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769ab36f-800a-4864-92eb-7ef45420ccc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:10.647362 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769ab36f-800a-4864-92eb-7ef45420ccc5 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:10.665056 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 993/1956] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:27:10.666244 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b7533-7684-433c-86f5-cbd0bf62be93 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:10.679773 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b7533-7684-433c-86f5-cbd0bf62be93 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:10.687323 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c145ff-867c-4f2c-9578-b27d7e6c7d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:10.688308 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c145ff-867c-4f2c-9578-b27d7e6c7d8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:10.688691 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c145ff-867c-4f2c-9578-b27d7e6c7d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:10.688955 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c145ff-867c-4f2c-9578-b27d7e6c7d8e admin admin] PATH_INFO: `/v3/projects/3b992135e3c54732b17ddc6aa0eeba29/users/f3b11a4660f249fcb6a11f4c6290e504/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:10.691634 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c145ff-867c-4f2c-9578-b27d7e6c7d8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b992135e3c54732b17ddc6aa0eeba29, user_id=f3b11a4660f249fcb6a11f4c6290e504, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:10.695892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c145ff-867c-4f2c-9578-b27d7e6c7d8e admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:10.707642 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 964/1957] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:27:10 2022] PUT /identity/v3/projects/3b992135e3c54732b17ddc6aa0eeba29/users/f3b11a4660f249fcb6a11f4c6290e504/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:27:10.718037 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14019443-c0c2-4c0e-a327-930df16b8d93 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:10.718358 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14019443-c0c2-4c0e-a327-930df16b8d93 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:10.718646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14019443-c0c2-4c0e-a327-930df16b8d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:10.754457 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-14019443-c0c2-4c0e-a327-930df16b8d93 None None] MFA Rules not processed for user `f3b11a4660f249fcb6a11f4c6290e504`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:10.755741 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-14019443-c0c2-4c0e-a327-930df16b8d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:10.829169 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 994/1958] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:10 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:27:10.840479 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c145ff-867c-4f2c-9578-b27d7e6c7d8e admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:10.901624 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c145ff-867c-4f2c-9578-b27d7e6c7d8e admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:10.905393 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee20567a-ddda-4bcd-9ddd-8e0336ca9879 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:10.906047 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20567a-ddda-4bcd-9ddd-8e0336ca9879 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:10.906340 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20567a-ddda-4bcd-9ddd-8e0336ca9879 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:10.906621 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20567a-ddda-4bcd-9ddd-8e0336ca9879 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:10.918277 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee20567a-ddda-4bcd-9ddd-8e0336ca9879 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:10.919728 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee20567a-ddda-4bcd-9ddd-8e0336ca9879 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:10.931789 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 965/1959] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:27:10 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:27:11.047403 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14019443-c0c2-4c0e-a327-930df16b8d93 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:11.049073 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-14019443-c0c2-4c0e-a327-930df16b8d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:11.062118 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14019443-c0c2-4c0e-a327-930df16b8d93 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:11.066202 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d756c803-5de7-498a-8c92-48caaafc18e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:11.066899 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756c803-5de7-498a-8c92-48caaafc18e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:11.067188 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756c803-5de7-498a-8c92-48caaafc18e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:11.067571 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756c803-5de7-498a-8c92-48caaafc18e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:11.124959 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:11.133314 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d756c803-5de7-498a-8c92-48caaafc18e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:11.135402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d756c803-5de7-498a-8c92-48caaafc18e3 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:11.145872 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 995/1960] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:27:11 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:27:14.020124 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9c398-0b98-490e-9c5d-9c5901a56817 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:14.021057 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9c398-0b98-490e-9c5d-9c5901a56817 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:14.021481 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9c398-0b98-490e-9c5d-9c5901a56817 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:14.023673 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 966/1961] 10.222.0.40 () {54 vars in 921 bytes} [Mon Jul 18 06:27:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:27:14.039289 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705cbc5-0e5f-4461-b996-e4a2c9272713 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:14.039806 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705cbc5-0e5f-4461-b996-e4a2c9272713 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:14.040292 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705cbc5-0e5f-4461-b996-e4a2c9272713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:14.085305 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-5705cbc5-0e5f-4461-b996-e4a2c9272713 None None] MFA Rules not processed for user `f92d948810ea4241810473506b654f7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:14.087916 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-5705cbc5-0e5f-4461-b996-e4a2c9272713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:14.198378 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 996/1962] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Jul 18 06:27:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:27:14.221926 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd9c398-0b98-490e-9c5d-9c5901a56817 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:14.223835 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-dfd9c398-0b98-490e-9c5d-9c5901a56817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:14.242214 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd9c398-0b98-490e-9c5d-9c5901a56817 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:14.252797 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5488c36-b19c-41e8-bcea-34fcf0d2790e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:14.253469 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5488c36-b19c-41e8-bcea-34fcf0d2790e service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:14.253752 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5488c36-b19c-41e8-bcea-34fcf0d2790e service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:14.254039 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5488c36-b19c-41e8-bcea-34fcf0d2790e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:14.265248 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5488c36-b19c-41e8-bcea-34fcf0d2790e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:14.266863 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5488c36-b19c-41e8-bcea-34fcf0d2790e service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:14.279827 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 967/1963] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:27:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:27:14.828776 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5705cbc5-0e5f-4461-b996-e4a2c9272713 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:14.840060 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5705cbc5-0e5f-4461-b996-e4a2c9272713 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:14.844467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2709ba67-1293-4cd4-b55b-a2095b42dea2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:14.846011 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2709ba67-1293-4cd4-b55b-a2095b42dea2 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:14.846011 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2709ba67-1293-4cd4-b55b-a2095b42dea2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:14.847093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2709ba67-1293-4cd4-b55b-a2095b42dea2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:14.954937 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2709ba67-1293-4cd4-b55b-a2095b42dea2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:14.957521 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2709ba67-1293-4cd4-b55b-a2095b42dea2 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:14.968283 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 997/1964] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:27:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:27:19.369148 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5488c36-b19c-41e8-bcea-34fcf0d2790e service nova] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:19.379445 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5488c36-b19c-41e8-bcea-34fcf0d2790e service nova] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:19.383142 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62206f59-c39c-4b11-b04f-ce5f3f56a329 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:19.383794 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62206f59-c39c-4b11-b04f-ce5f3f56a329 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:19.384080 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62206f59-c39c-4b11-b04f-ce5f3f56a329 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:19.384370 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62206f59-c39c-4b11-b04f-ce5f3f56a329 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:19.451044 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62206f59-c39c-4b11-b04f-ce5f3f56a329 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:19.452522 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62206f59-c39c-4b11-b04f-ce5f3f56a329 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:19.466227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 968/1965] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:27:19 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:27:21.432879 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2709ba67-1293-4cd4-b55b-a2095b42dea2 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:21.509666 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2709ba67-1293-4cd4-b55b-a2095b42dea2 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:21.515795 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67593f22-55c8-4d90-a027-73b42acbc6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:21.516750 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67593f22-55c8-4d90-a027-73b42acbc6b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:21.517033 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67593f22-55c8-4d90-a027-73b42acbc6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:21.517811 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67593f22-55c8-4d90-a027-73b42acbc6b7 admin admin] PATH_INFO: `/v3/users/8b5a3e1b06d34993babc9a95d5a3f1ab` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:21.519558 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67593f22-55c8-4d90-a027-73b42acbc6b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b5a3e1b06d34993babc9a95d5a3f1ab)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:21.521570 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67593f22-55c8-4d90-a027-73b42acbc6b7 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:21.724689 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:21.810388 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62206f59-c39c-4b11-b04f-ce5f3f56a329 service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:21.812451 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-62206f59-c39c-4b11-b04f-ce5f3f56a329 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:21.832604 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-67593f22-55c8-4d90-a027-73b42acbc6b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b5a3e1b06d34993babc9a95d5a3f1ab'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:21.840150 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-67593f22-55c8-4d90-a027-73b42acbc6b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b5a3e1b06d34993babc9a95d5a3f1ab'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:21.841343 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-67593f22-55c8-4d90-a027-73b42acbc6b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b5a3e1b06d34993babc9a95d5a3f1ab'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:21.842208 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-67593f22-55c8-4d90-a027-73b42acbc6b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b5a3e1b06d34993babc9a95d5a3f1ab'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:21.848459 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62206f59-c39c-4b11-b04f-ce5f3f56a329 service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:21.865592 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff73fc1-a7d0-4f7d-8232-c4571da9cb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:21.865592 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff73fc1-a7d0-4f7d-8232-c4571da9cb7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:21.865592 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff73fc1-a7d0-4f7d-8232-c4571da9cb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:21.865592 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff73fc1-a7d0-4f7d-8232-c4571da9cb7e admin admin] PATH_INFO: `/v3/users/9797166c6ef84691a191d88dc065fff6` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:21.866111 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff73fc1-a7d0-4f7d-8232-c4571da9cb7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9797166c6ef84691a191d88dc065fff6)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:21.866111 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff73fc1-a7d0-4f7d-8232-c4571da9cb7e admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:21.870737 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-67593f22-55c8-4d90-a027-73b42acbc6b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b5a3e1b06d34993babc9a95d5a3f1ab'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:21.979597 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 998/1966] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:27:21 2022] DELETE /identity/v3/users/8b5a3e1b06d34993babc9a95d5a3f1ab => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:27:21.985537 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5cf306-beba-4f41-8287-436b95facaf6 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:21.986637 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5cf306-beba-4f41-8287-436b95facaf6 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:21.987447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5cf306-beba-4f41-8287-436b95facaf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:22.029879 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-bb5cf306-beba-4f41-8287-436b95facaf6 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:22.031227 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-bb5cf306-beba-4f41-8287-436b95facaf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:22.117008 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0ff73fc1-a7d0-4f7d-8232-c4571da9cb7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9797166c6ef84691a191d88dc065fff6'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:22.125177 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 999/1967] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:27:22.131431 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:22.133933 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0ff73fc1-a7d0-4f7d-8232-c4571da9cb7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9797166c6ef84691a191d88dc065fff6'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:22.136799 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5cf306-beba-4f41-8287-436b95facaf6 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:22.148016 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-bb5cf306-beba-4f41-8287-436b95facaf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:22.183162 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0ff73fc1-a7d0-4f7d-8232-c4571da9cb7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9797166c6ef84691a191d88dc065fff6'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:22.185296 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0ff73fc1-a7d0-4f7d-8232-c4571da9cb7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9797166c6ef84691a191d88dc065fff6'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:22.186240 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0ff73fc1-a7d0-4f7d-8232-c4571da9cb7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9797166c6ef84691a191d88dc065fff6'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:22.249522 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5cf306-beba-4f41-8287-436b95facaf6 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:22.255376 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc9ecd-43ae-4043-b114-08da44a136e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:22.256507 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc9ecd-43ae-4043-b114-08da44a136e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:22.257011 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc9ecd-43ae-4043-b114-08da44a136e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:22.257362 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc9ecd-43ae-4043-b114-08da44a136e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:22.260646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-b3cc9ecd-43ae-4043-b114-08da44a136e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:22.278579 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc9ecd-43ae-4043-b114-08da44a136e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:22.282279 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc9ecd-43ae-4043-b114-08da44a136e8 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:22.292208 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1000/1968] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:27:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 06:27:22.319050 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 969/1969] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:27:21 2022] DELETE /identity/v3/users/9797166c6ef84691a191d88dc065fff6 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:27:22.319453 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f982e6e-0511-40fb-979c-704d158c7e25 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:22.319751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f982e6e-0511-40fb-979c-704d158c7e25 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:22.320032 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f982e6e-0511-40fb-979c-704d158c7e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:22.339762 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-0f982e6e-0511-40fb-979c-704d158c7e25 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:22.341022 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-0f982e6e-0511-40fb-979c-704d158c7e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:22.404343 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:22.427149 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1001/1970] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:27:22.437929 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff73fc1-a7d0-4f7d-8232-c4571da9cb7e admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:22.440250 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-0ff73fc1-a7d0-4f7d-8232-c4571da9cb7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:22.510018 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff73fc1-a7d0-4f7d-8232-c4571da9cb7e admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:22.513615 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd1b8e-a136-46b1-bd41-0bdf3e0a9d5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:22.514271 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd1b8e-a136-46b1-bd41-0bdf3e0a9d5b service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:22.514559 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd1b8e-a136-46b1-bd41-0bdf3e0a9d5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:22.514842 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd1b8e-a136-46b1-bd41-0bdf3e0a9d5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:22.524242 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd1b8e-a136-46b1-bd41-0bdf3e0a9d5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:22.525709 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd1b8e-a136-46b1-bd41-0bdf3e0a9d5b service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:22.536049 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 970/1971] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:27:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:27:22.858669 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f982e6e-0511-40fb-979c-704d158c7e25 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:22.860546 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-0f982e6e-0511-40fb-979c-704d158c7e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:22.875910 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f982e6e-0511-40fb-979c-704d158c7e25 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:22.884084 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c4b08-cca8-4861-92c8-0023406ecdc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:22.885320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c4b08-cca8-4861-92c8-0023406ecdc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:22.885694 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c4b08-cca8-4861-92c8-0023406ecdc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:22.886052 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c4b08-cca8-4861-92c8-0023406ecdc7 admin admin] PATH_INFO: `/v3/projects/5734791ff2694a5eafc455e79878520a` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:22.887529 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563c4b08-cca8-4861-92c8-0023406ecdc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5734791ff2694a5eafc455e79878520a)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:22.889796 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563c4b08-cca8-4861-92c8-0023406ecdc7 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:23.010644 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-563c4b08-cca8-4861-92c8-0023406ecdc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5734791ff2694a5eafc455e79878520a', 'request_id': 'req-563c4b08-cca8-4861-92c8-0023406ecdc7'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:23.019315 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-563c4b08-cca8-4861-92c8-0023406ecdc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5734791ff2694a5eafc455e79878520a', 'request_id': 'req-563c4b08-cca8-4861-92c8-0023406ecdc7'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:23.045072 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd1b8e-a136-46b1-bd41-0bdf3e0a9d5b service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:23.046729 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-bacd1b8e-a136-46b1-bd41-0bdf3e0a9d5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:23.063038 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd1b8e-a136-46b1-bd41-0bdf3e0a9d5b service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:23.123241 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-563c4b08-cca8-4861-92c8-0023406ecdc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5734791ff2694a5eafc455e79878520a', 'request_id': 'req-563c4b08-cca8-4861-92c8-0023406ecdc7'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:23.129142 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e887f3-8298-4e66-bde7-0debf9a7810c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:23.130016 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e887f3-8298-4e66-bde7-0debf9a7810c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:23.130304 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e887f3-8298-4e66-bde7-0debf9a7810c admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:23.130588 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e887f3-8298-4e66-bde7-0debf9a7810c admin admin] PATH_INFO: `/v3/projects/a5fcebce62564cab9e0a4450d93670ee` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:23.131871 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e887f3-8298-4e66-bde7-0debf9a7810c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5fcebce62564cab9e0a4450d93670ee)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:23.134045 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e887f3-8298-4e66-bde7-0debf9a7810c admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:23.135400 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-563c4b08-cca8-4861-92c8-0023406ecdc7 admin admin] The token cache is being invalidate because project 5734791ff2694a5eafc455e79878520a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:27:23.135771 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-563c4b08-cca8-4861-92c8-0023406ecdc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:23.136437 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-563c4b08-cca8-4861-92c8-0023406ecdc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:23.138228 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1002/1972] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:27:22 2022] DELETE /identity/v3/projects/5734791ff2694a5eafc455e79878520a => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:27:23.141928 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c4b08-cca8-4861-92c8-0023406ecdc7 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:23.144958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-563c4b08-cca8-4861-92c8-0023406ecdc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:23.159935 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c4b08-cca8-4861-92c8-0023406ecdc7 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:23.165015 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9fa524-e5da-4ab9-bc0c-5f6900dd3aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:23.165894 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9fa524-e5da-4ab9-bc0c-5f6900dd3aa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:23.166176 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9fa524-e5da-4ab9-bc0c-5f6900dd3aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:23.166455 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9fa524-e5da-4ab9-bc0c-5f6900dd3aa2 admin admin] PATH_INFO: `/v3/users/607d6f7410eb477e97136cf6d85bedbb` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:23.167654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9fa524-e5da-4ab9-bc0c-5f6900dd3aa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=607d6f7410eb477e97136cf6d85bedbb)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:23.169669 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9fa524-e5da-4ab9-bc0c-5f6900dd3aa2 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:23.277760 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-33e887f3-8298-4e66-bde7-0debf9a7810c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5fcebce62564cab9e0a4450d93670ee', 'request_id': 'req-33e887f3-8298-4e66-bde7-0debf9a7810c'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:23.284374 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-33e887f3-8298-4e66-bde7-0debf9a7810c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5fcebce62564cab9e0a4450d93670ee', 'request_id': 'req-33e887f3-8298-4e66-bde7-0debf9a7810c'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:23.392139 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-33e887f3-8298-4e66-bde7-0debf9a7810c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5fcebce62564cab9e0a4450d93670ee', 'request_id': 'req-33e887f3-8298-4e66-bde7-0debf9a7810c'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:23.404996 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-33e887f3-8298-4e66-bde7-0debf9a7810c admin admin] The token cache is being invalidate because project a5fcebce62564cab9e0a4450d93670ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:27:23.405362 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-33e887f3-8298-4e66-bde7-0debf9a7810c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:23.406013 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-33e887f3-8298-4e66-bde7-0debf9a7810c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:23.407685 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 971/1973] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:27:23 2022] DELETE /identity/v3/projects/a5fcebce62564cab9e0a4450d93670ee => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:27:23.414242 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd6d5e-d5e0-4e72-9c94-6b9bb3deeb58 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:23.414539 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd6d5e-d5e0-4e72-9c94-6b9bb3deeb58 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:23.414824 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd6d5e-d5e0-4e72-9c94-6b9bb3deeb58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:23.424362 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:23.437724 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-acdd6d5e-d5e0-4e72-9c94-6b9bb3deeb58 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:23.438901 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-acdd6d5e-d5e0-4e72-9c94-6b9bb3deeb58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:23.439264 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6d9fa524-e5da-4ab9-bc0c-5f6900dd3aa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '607d6f7410eb477e97136cf6d85bedbb'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:23.447994 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6d9fa524-e5da-4ab9-bc0c-5f6900dd3aa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '607d6f7410eb477e97136cf6d85bedbb'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:23.453204 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6d9fa524-e5da-4ab9-bc0c-5f6900dd3aa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '607d6f7410eb477e97136cf6d85bedbb'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:23.453204 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6d9fa524-e5da-4ab9-bc0c-5f6900dd3aa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '607d6f7410eb477e97136cf6d85bedbb'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:23.471482 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6d9fa524-e5da-4ab9-bc0c-5f6900dd3aa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '607d6f7410eb477e97136cf6d85bedbb'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:23.513574 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 972/1974] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:27:23.521503 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffedae-c8f8-4115-af0c-35f13116c596 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:23.522089 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffedae-c8f8-4115-af0c-35f13116c596 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:23.522649 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffedae-c8f8-4115-af0c-35f13116c596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:23.550642 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-f6ffedae-c8f8-4115-af0c-35f13116c596 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:23.550642 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f6ffedae-c8f8-4115-af0c-35f13116c596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:23.556396 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 973/1975] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:27:23.567631 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ffedae-c8f8-4115-af0c-35f13116c596 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:23.569355 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1003/1976] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:27:23 2022] DELETE /identity/v3/users/607d6f7410eb477e97136cf6d85bedbb => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:27:23.582433 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ffedae-c8f8-4115-af0c-35f13116c596 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:23.659232 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cfa1ed-d7c1-469b-be6b-8761dc217780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:23.659924 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cfa1ed-d7c1-469b-be6b-8761dc217780 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:23.660214 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cfa1ed-d7c1-469b-be6b-8761dc217780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:23.660527 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cfa1ed-d7c1-469b-be6b-8761dc217780 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:23.661275 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cfa1ed-d7c1-469b-be6b-8761dc217780 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:27:23.661586 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cfa1ed-d7c1-469b-be6b-8761dc217780 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:23.663323 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cfa1ed-d7c1-469b-be6b-8761dc217780 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:23.677903 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 974/1977] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:27:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:27:23.687472 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9fa524-e5da-4ab9-bc0c-5f6900dd3aa2 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:23.699049 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9fa524-e5da-4ab9-bc0c-5f6900dd3aa2 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:23.706892 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e38f7-bdd6-4e7b-9de2-91cf28b9cd0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:23.707664 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e38f7-bdd6-4e7b-9de2-91cf28b9cd0f admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:23.707946 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e38f7-bdd6-4e7b-9de2-91cf28b9cd0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:23.708241 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e38f7-bdd6-4e7b-9de2-91cf28b9cd0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:23.708984 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e38f7-bdd6-4e7b-9de2-91cf28b9cd0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:23.711039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e38f7-bdd6-4e7b-9de2-91cf28b9cd0f admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:23.780014 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1004/1978] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:27:23 2022] POST /identity/v3/projects => generated 385 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:27:23.784862 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cfa1ed-d7c1-469b-be6b-8761dc217780 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:23.794915 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cfa1ed-d7c1-469b-be6b-8761dc217780 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:23.855363 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4536b34-db38-4310-8925-ebf057f21d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:23.856064 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4536b34-db38-4310-8925-ebf057f21d85 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:23.856337 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4536b34-db38-4310-8925-ebf057f21d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:23.856743 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4536b34-db38-4310-8925-ebf057f21d85 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:23.857514 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4536b34-db38-4310-8925-ebf057f21d85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:23.863020 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4536b34-db38-4310-8925-ebf057f21d85 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:23.954298 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 975/1979] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:27:23 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:27:23.955859 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e38f7-bdd6-4e7b-9de2-91cf28b9cd0f admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:23.966561 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e38f7-bdd6-4e7b-9de2-91cf28b9cd0f admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:23.971235 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d6f756-b9f6-43b5-806c-9f10eea72065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:23.971944 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d6f756-b9f6-43b5-806c-9f10eea72065 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:23.972231 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d6f756-b9f6-43b5-806c-9f10eea72065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:23.972558 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d6f756-b9f6-43b5-806c-9f10eea72065 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:23.973286 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d6f756-b9f6-43b5-806c-9f10eea72065 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:23.974896 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d6f756-b9f6-43b5-806c-9f10eea72065 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:24.006536 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1005/1980] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:27:24.009907 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4536b34-db38-4310-8925-ebf057f21d85 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:24.020757 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4536b34-db38-4310-8925-ebf057f21d85 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:24.026647 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92d6d9c-c714-4bcd-bda9-24c4f1a849e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:24.027329 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d6d9c-c714-4bcd-bda9-24c4f1a849e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:24.027611 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d6d9c-c714-4bcd-bda9-24c4f1a849e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:24.027894 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d6d9c-c714-4bcd-bda9-24c4f1a849e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:24.028665 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92d6d9c-c714-4bcd-bda9-24c4f1a849e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:24.030343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92d6d9c-c714-4bcd-bda9-24c4f1a849e5 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:24.043062 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 976/1981] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:27:24.047223 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d6f756-b9f6-43b5-806c-9f10eea72065 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:24.060357 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d6f756-b9f6-43b5-806c-9f10eea72065 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:24.067234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcde202-e154-4f9f-b757-800e8d7b8f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:24.069162 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcde202-e154-4f9f-b757-800e8d7b8f2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:24.069162 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcde202-e154-4f9f-b757-800e8d7b8f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:24.069162 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcde202-e154-4f9f-b757-800e8d7b8f2e admin admin] PATH_INFO: `/v3/projects/257b82c51e534cd98b484351a11a17da/users/09791fda27424feda8db22ba0b481e1b/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:24.072823 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:24.092066 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcde202-e154-4f9f-b757-800e8d7b8f2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=257b82c51e534cd98b484351a11a17da, user_id=09791fda27424feda8db22ba0b481e1b, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:24.094880 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcde202-e154-4f9f-b757-800e8d7b8f2e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:24.142130 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92d6d9c-c714-4bcd-bda9-24c4f1a849e5 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:24.144019 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-c92d6d9c-c714-4bcd-bda9-24c4f1a849e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:24.157319 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92d6d9c-c714-4bcd-bda9-24c4f1a849e5 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:24.165066 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937fbd36-4520-41a0-8732-52c0c5b417a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:24.166226 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fbd36-4520-41a0-8732-52c0c5b417a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:24.166610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fbd36-4520-41a0-8732-52c0c5b417a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:24.166848 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fbd36-4520-41a0-8732-52c0c5b417a1 admin admin] PATH_INFO: `/v3/projects/d544338ddcc147fa900a140eeeb5ff87` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:24.168323 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937fbd36-4520-41a0-8732-52c0c5b417a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d544338ddcc147fa900a140eeeb5ff87)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:24.170597 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937fbd36-4520-41a0-8732-52c0c5b417a1 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:24.180107 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:24.245626 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1006/1982] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:27:24 2022] PUT /identity/v3/projects/257b82c51e534cd98b484351a11a17da/users/09791fda27424feda8db22ba0b481e1b/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:27:24.257630 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7750d7d-d70b-42bc-83cc-85319f3a3513 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:24.257924 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7750d7d-d70b-42bc-83cc-85319f3a3513 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:24.258202 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7750d7d-d70b-42bc-83cc-85319f3a3513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:24.287718 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-f7750d7d-d70b-42bc-83cc-85319f3a3513 None None] MFA Rules not processed for user `09791fda27424feda8db22ba0b481e1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:24.289083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-f7750d7d-d70b-42bc-83cc-85319f3a3513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:24.363302 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-937fbd36-4520-41a0-8732-52c0c5b417a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd544338ddcc147fa900a140eeeb5ff87', 'request_id': 'req-937fbd36-4520-41a0-8732-52c0c5b417a1'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:24.370922 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-937fbd36-4520-41a0-8732-52c0c5b417a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd544338ddcc147fa900a140eeeb5ff87', 'request_id': 'req-937fbd36-4520-41a0-8732-52c0c5b417a1'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:24.388971 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1007/1983] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:27:24.440440 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7750d7d-d70b-42bc-83cc-85319f3a3513 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:24.441927 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-f7750d7d-d70b-42bc-83cc-85319f3a3513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:24.449356 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-937fbd36-4520-41a0-8732-52c0c5b417a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd544338ddcc147fa900a140eeeb5ff87', 'request_id': 'req-937fbd36-4520-41a0-8732-52c0c5b417a1'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:24.467969 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-937fbd36-4520-41a0-8732-52c0c5b417a1 admin admin] The token cache is being invalidate because project d544338ddcc147fa900a140eeeb5ff87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:27:24.468281 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-937fbd36-4520-41a0-8732-52c0c5b417a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:24.468999 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-937fbd36-4520-41a0-8732-52c0c5b417a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:24.471594 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 977/1984] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:27:24 2022] DELETE /identity/v3/projects/d544338ddcc147fa900a140eeeb5ff87 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:27:24.475493 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937fbd36-4520-41a0-8732-52c0c5b417a1 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:24.476957 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-937fbd36-4520-41a0-8732-52c0c5b417a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:24.511026 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7750d7d-d70b-42bc-83cc-85319f3a3513 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:24.545839 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937fbd36-4520-41a0-8732-52c0c5b417a1 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:24.552994 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd0e10-8fab-498c-a500-6cdcc2c9a1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:24.553952 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd0e10-8fab-498c-a500-6cdcc2c9a1db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:24.554230 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd0e10-8fab-498c-a500-6cdcc2c9a1db admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:24.554508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd0e10-8fab-498c-a500-6cdcc2c9a1db admin admin] PATH_INFO: `/v3/users/f3b11a4660f249fcb6a11f4c6290e504` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:24.555878 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fd0e10-8fab-498c-a500-6cdcc2c9a1db admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3b11a4660f249fcb6a11f4c6290e504)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:24.557812 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fd0e10-8fab-498c-a500-6cdcc2c9a1db admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:24.563845 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e2178-6d5c-43e4-935a-c383a4ba93c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:24.566886 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e2178-6d5c-43e4-935a-c383a4ba93c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:24.566886 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e2178-6d5c-43e4-935a-c383a4ba93c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:24.566886 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e2178-6d5c-43e4-935a-c383a4ba93c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:24.576698 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-fd9e2178-6d5c-43e4-935a-c383a4ba93c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:24.639957 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:24.656818 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e2178-6d5c-43e4-935a-c383a4ba93c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:24.663269 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e2178-6d5c-43e4-935a-c383a4ba93c4 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:24.673756 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1008/1985] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:27:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:27:24.782421 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f2fd0e10-8fab-498c-a500-6cdcc2c9a1db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3b11a4660f249fcb6a11f4c6290e504'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:24.793659 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f2fd0e10-8fab-498c-a500-6cdcc2c9a1db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3b11a4660f249fcb6a11f4c6290e504'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:24.812379 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f2fd0e10-8fab-498c-a500-6cdcc2c9a1db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3b11a4660f249fcb6a11f4c6290e504'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:24.813272 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f2fd0e10-8fab-498c-a500-6cdcc2c9a1db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3b11a4660f249fcb6a11f4c6290e504'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:24.814225 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f2fd0e10-8fab-498c-a500-6cdcc2c9a1db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3b11a4660f249fcb6a11f4c6290e504'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:24.878921 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 978/1986] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:27:24 2022] DELETE /identity/v3/users/f3b11a4660f249fcb6a11f4c6290e504 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:27:25.408491 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e2178-6d5c-43e4-935a-c383a4ba93c4 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:25.419812 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e2178-6d5c-43e4-935a-c383a4ba93c4 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:25.494494 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b6c563-dccf-4569-a94e-d247714b862b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:25.495406 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b6c563-dccf-4569-a94e-d247714b862b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:25.495691 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b6c563-dccf-4569-a94e-d247714b862b admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:25.495980 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b6c563-dccf-4569-a94e-d247714b862b admin admin] PATH_INFO: `/v3/projects/3b992135e3c54732b17ddc6aa0eeba29` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:25.497218 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b6c563-dccf-4569-a94e-d247714b862b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b992135e3c54732b17ddc6aa0eeba29)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:25.499129 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b6c563-dccf-4569-a94e-d247714b862b admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:25.628399 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-54b6c563-dccf-4569-a94e-d247714b862b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b992135e3c54732b17ddc6aa0eeba29', 'request_id': 'req-54b6c563-dccf-4569-a94e-d247714b862b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:25.635256 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-54b6c563-dccf-4569-a94e-d247714b862b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b992135e3c54732b17ddc6aa0eeba29', 'request_id': 'req-54b6c563-dccf-4569-a94e-d247714b862b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:25.734903 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-54b6c563-dccf-4569-a94e-d247714b862b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b992135e3c54732b17ddc6aa0eeba29', 'request_id': 'req-54b6c563-dccf-4569-a94e-d247714b862b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:25.747062 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-54b6c563-dccf-4569-a94e-d247714b862b admin admin] The token cache is being invalidate because project 3b992135e3c54732b17ddc6aa0eeba29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:27:25.747358 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-54b6c563-dccf-4569-a94e-d247714b862b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:25.748022 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-54b6c563-dccf-4569-a94e-d247714b862b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:25.749644 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1009/1987] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:27:25 2022] DELETE /identity/v3/projects/3b992135e3c54732b17ddc6aa0eeba29 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:27:25.755982 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f412ec-587c-4c0d-a6f5-78e49ceeed32 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:25.756553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f412ec-587c-4c0d-a6f5-78e49ceeed32 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:25.756655 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f412ec-587c-4c0d-a6f5-78e49ceeed32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:25.774474 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-e5f412ec-587c-4c0d-a6f5-78e49ceeed32 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:25.775676 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e5f412ec-587c-4c0d-a6f5-78e49ceeed32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:25.784607 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:25.836260 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 979/1988] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:27:25.844974 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53708252-7782-4c7f-9ce7-9479e880b0fd None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:25.845328 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53708252-7782-4c7f-9ce7-9479e880b0fd None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:25.845618 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53708252-7782-4c7f-9ce7-9479e880b0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:25.864632 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-53708252-7782-4c7f-9ce7-9479e880b0fd None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:25.866328 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-53708252-7782-4c7f-9ce7-9479e880b0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:25.876211 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1010/1989] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:27:25.880980 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f412ec-587c-4c0d-a6f5-78e49ceeed32 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:25.890889 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f412ec-587c-4c0d-a6f5-78e49ceeed32 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:25.911843 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2514d1b-dec4-47a4-b3b1-9e46430cb88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:25.916577 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2514d1b-dec4-47a4-b3b1-9e46430cb88e admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:25.916878 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2514d1b-dec4-47a4-b3b1-9e46430cb88e admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:25.917176 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2514d1b-dec4-47a4-b3b1-9e46430cb88e admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:25.918089 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2514d1b-dec4-47a4-b3b1-9e46430cb88e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:27:25.918256 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2514d1b-dec4-47a4-b3b1-9e46430cb88e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:25.919881 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2514d1b-dec4-47a4-b3b1-9e46430cb88e admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:25.940062 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 980/1990] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:27:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:27:25.944989 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53708252-7782-4c7f-9ce7-9479e880b0fd None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:25.957609 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53708252-7782-4c7f-9ce7-9479e880b0fd None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:25.963103 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0c0b2-d80e-4bf9-bd8f-6a2df9fe75c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:25.963862 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0c0b2-d80e-4bf9-bd8f-6a2df9fe75c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:25.964343 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0c0b2-d80e-4bf9-bd8f-6a2df9fe75c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:25.964723 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0c0b2-d80e-4bf9-bd8f-6a2df9fe75c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:25.965504 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de0c0b2-d80e-4bf9-bd8f-6a2df9fe75c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:25.967542 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de0c0b2-d80e-4bf9-bd8f-6a2df9fe75c5 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:26.074389 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1011/1991] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:27:25 2022] POST /identity/v3/projects => generated 383 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:27:26.078793 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2514d1b-dec4-47a4-b3b1-9e46430cb88e admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:26.090239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2514d1b-dec4-47a4-b3b1-9e46430cb88e admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:26.153437 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318753c5-4968-4bdf-bd3b-05b19a0aed03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:26.154145 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318753c5-4968-4bdf-bd3b-05b19a0aed03 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:26.154429 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318753c5-4968-4bdf-bd3b-05b19a0aed03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:26.154707 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318753c5-4968-4bdf-bd3b-05b19a0aed03 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:26.155418 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318753c5-4968-4bdf-bd3b-05b19a0aed03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:26.157375 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318753c5-4968-4bdf-bd3b-05b19a0aed03 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:26.266844 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 981/1992] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:27:26 2022] POST /identity/v3/users => generated 292 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:27:26.271107 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0c0b2-d80e-4bf9-bd8f-6a2df9fe75c5 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:26.282037 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0c0b2-d80e-4bf9-bd8f-6a2df9fe75c5 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:26.288391 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80593477-b29b-44a2-b179-864944ef19e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:26.289135 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80593477-b29b-44a2-b179-864944ef19e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:26.289417 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80593477-b29b-44a2-b179-864944ef19e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:26.289694 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80593477-b29b-44a2-b179-864944ef19e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:26.290504 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80593477-b29b-44a2-b179-864944ef19e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:26.292140 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80593477-b29b-44a2-b179-864944ef19e5 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:26.304256 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1012/1993] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:27:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:27:26.311463 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318753c5-4968-4bdf-bd3b-05b19a0aed03 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:26.340770 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318753c5-4968-4bdf-bd3b-05b19a0aed03 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:26.346581 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c8d91-e3d0-45e8-b284-703aa17e66df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:26.347273 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c8d91-e3d0-45e8-b284-703aa17e66df admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:26.347558 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c8d91-e3d0-45e8-b284-703aa17e66df admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:26.347844 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c8d91-e3d0-45e8-b284-703aa17e66df admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:26.348873 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c8d91-e3d0-45e8-b284-703aa17e66df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:26.350577 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c8d91-e3d0-45e8-b284-703aa17e66df admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:26.363185 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 982/1994] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:27:26 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:27:26.368513 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80593477-b29b-44a2-b179-864944ef19e5 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:26.380543 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80593477-b29b-44a2-b179-864944ef19e5 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:26.385689 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8d23c-c921-47ad-a709-d559d0f0b6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:26.387191 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8d23c-c921-47ad-a709-d559d0f0b6bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:26.387486 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8d23c-c921-47ad-a709-d559d0f0b6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:26.387900 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8d23c-c921-47ad-a709-d559d0f0b6bd admin admin] PATH_INFO: `/v3/projects/715a4d0115fb4910b94957cf64a632fd/users/dce08a4dda9e4e788f766d58f18df587/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:26.404712 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada8d23c-c921-47ad-a709-d559d0f0b6bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=715a4d0115fb4910b94957cf64a632fd, user_id=dce08a4dda9e4e788f766d58f18df587, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:26.407430 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada8d23c-c921-47ad-a709-d559d0f0b6bd admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:26.483921 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1013/1995] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:27:26 2022] PUT /identity/v3/projects/715a4d0115fb4910b94957cf64a632fd/users/dce08a4dda9e4e788f766d58f18df587/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:27:26.497138 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4a77a-e3c8-4099-8777-fe830d2ed3f7 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:26.498089 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4a77a-e3c8-4099-8777-fe830d2ed3f7 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:26.498089 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4a77a-e3c8-4099-8777-fe830d2ed3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:26.525430 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-a3c4a77a-e3c8-4099-8777-fe830d2ed3f7 None None] MFA Rules not processed for user `dce08a4dda9e4e788f766d58f18df587`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:26.526651 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a3c4a77a-e3c8-4099-8777-fe830d2ed3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:26.592505 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 983/1996] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:27:26.604340 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8d23c-c921-47ad-a709-d559d0f0b6bd admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:26.606097 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ada8d23c-c921-47ad-a709-d559d0f0b6bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:26.620311 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:26.663948 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8d23c-c921-47ad-a709-d559d0f0b6bd admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:26.667580 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2338d9-e781-40f9-bed1-c57adedece2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:26.668236 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2338d9-e781-40f9-bed1-c57adedece2e service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:26.668568 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2338d9-e781-40f9-bed1-c57adedece2e service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:26.668848 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2338d9-e781-40f9-bed1-c57adedece2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:26.678523 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2338d9-e781-40f9-bed1-c57adedece2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:26.679962 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2338d9-e781-40f9-bed1-c57adedece2e service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:26.689793 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1014/1997] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:27:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:27:26.709428 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4a77a-e3c8-4099-8777-fe830d2ed3f7 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:26.719003 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:26.771965 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4a77a-e3c8-4099-8777-fe830d2ed3f7 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:26.776749 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e9e8b3-004f-4e80-a121-c3df6d54095a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:26.777635 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9e8b3-004f-4e80-a121-c3df6d54095a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:26.777924 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9e8b3-004f-4e80-a121-c3df6d54095a admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:26.778203 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9e8b3-004f-4e80-a121-c3df6d54095a admin admin] PATH_INFO: `/v3/users/dce08a4dda9e4e788f766d58f18df587` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:26.779585 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e9e8b3-004f-4e80-a121-c3df6d54095a admin admin] RBAC: Authorizing `identity:delete_user(user_id=dce08a4dda9e4e788f766d58f18df587)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:26.782121 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e9e8b3-004f-4e80-a121-c3df6d54095a admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:26.970810 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f4e9e8b3-004f-4e80-a121-c3df6d54095a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dce08a4dda9e4e788f766d58f18df587'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:26.977903 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f4e9e8b3-004f-4e80-a121-c3df6d54095a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dce08a4dda9e4e788f766d58f18df587'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:26.991586 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f4e9e8b3-004f-4e80-a121-c3df6d54095a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dce08a4dda9e4e788f766d58f18df587'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:26.992390 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f4e9e8b3-004f-4e80-a121-c3df6d54095a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dce08a4dda9e4e788f766d58f18df587'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:26.993074 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f4e9e8b3-004f-4e80-a121-c3df6d54095a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dce08a4dda9e4e788f766d58f18df587'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:27.060677 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 984/1998] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:27:26 2022] DELETE /identity/v3/users/dce08a4dda9e4e788f766d58f18df587 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:27:27.066030 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f5ce9-7d18-4f2e-b120-e3ba237d8f19 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:27.066279 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f5ce9-7d18-4f2e-b120-e3ba237d8f19 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:27.066562 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f5ce9-7d18-4f2e-b120-e3ba237d8f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:27.083693 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-4f1f5ce9-7d18-4f2e-b120-e3ba237d8f19 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:27.084989 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4f1f5ce9-7d18-4f2e-b120-e3ba237d8f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:27.157233 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1015/1999] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:27:27.167552 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e9e8b3-004f-4e80-a121-c3df6d54095a admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:27.169187 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f4e9e8b3-004f-4e80-a121-c3df6d54095a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:27.229548 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e9e8b3-004f-4e80-a121-c3df6d54095a admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:27.233180 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba4f9e-8bfd-4437-8503-97412ba25868 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:27.233826 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba4f9e-8bfd-4437-8503-97412ba25868 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:27.234116 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba4f9e-8bfd-4437-8503-97412ba25868 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:27.234401 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba4f9e-8bfd-4437-8503-97412ba25868 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:27.245210 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ba4f9e-8bfd-4437-8503-97412ba25868 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:27.246696 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ba4f9e-8bfd-4437-8503-97412ba25868 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:27.257694 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 985/2000] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:27:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:27:27.926329 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f5ce9-7d18-4f2e-b120-e3ba237d8f19 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:27.928908 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4f1f5ce9-7d18-4f2e-b120-e3ba237d8f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:27.947494 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f5ce9-7d18-4f2e-b120-e3ba237d8f19 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:27.953389 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1490cb-fb10-4f8c-aec8-969e26864f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:27.954951 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1490cb-fb10-4f8c-aec8-969e26864f17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:27.955250 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1490cb-fb10-4f8c-aec8-969e26864f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:27.955773 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1490cb-fb10-4f8c-aec8-969e26864f17 admin admin] PATH_INFO: `/v3/projects/715a4d0115fb4910b94957cf64a632fd` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:27.957591 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1490cb-fb10-4f8c-aec8-969e26864f17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=715a4d0115fb4910b94957cf64a632fd)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:27.959999 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1490cb-fb10-4f8c-aec8-969e26864f17 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:28.124336 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ee1490cb-fb10-4f8c-aec8-969e26864f17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '715a4d0115fb4910b94957cf64a632fd', 'request_id': 'req-ee1490cb-fb10-4f8c-aec8-969e26864f17'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:28.132245 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ee1490cb-fb10-4f8c-aec8-969e26864f17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '715a4d0115fb4910b94957cf64a632fd', 'request_id': 'req-ee1490cb-fb10-4f8c-aec8-969e26864f17'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:28.219928 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ee1490cb-fb10-4f8c-aec8-969e26864f17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '715a4d0115fb4910b94957cf64a632fd', 'request_id': 'req-ee1490cb-fb10-4f8c-aec8-969e26864f17'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:28.232072 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ee1490cb-fb10-4f8c-aec8-969e26864f17 admin admin] The token cache is being invalidate because project 715a4d0115fb4910b94957cf64a632fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:27:28.232387 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ee1490cb-fb10-4f8c-aec8-969e26864f17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:28.233058 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ee1490cb-fb10-4f8c-aec8-969e26864f17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:28.234786 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1016/2001] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:27:27 2022] DELETE /identity/v3/projects/715a4d0115fb4910b94957cf64a632fd => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:27:28.240437 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e54572-2537-4afd-a1b5-645df4069ae6 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:28.240959 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e54572-2537-4afd-a1b5-645df4069ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:28.241057 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e54572-2537-4afd-a1b5-645df4069ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:28.259365 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-f9e54572-2537-4afd-a1b5-645df4069ae6 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:28.260612 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f9e54572-2537-4afd-a1b5-645df4069ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:28.291194 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:28.325612 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 986/2002] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:27:28.334415 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d93d3-371e-49b4-9ade-6a22fedf5c7f None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:28.334693 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d93d3-371e-49b4-9ade-6a22fedf5c7f None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:28.334971 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d93d3-371e-49b4-9ade-6a22fedf5c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:28.356638 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-479d93d3-371e-49b4-9ade-6a22fedf5c7f None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:28.357803 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-479d93d3-371e-49b4-9ade-6a22fedf5c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:28.366908 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1017/2003] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:27:28.371893 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e54572-2537-4afd-a1b5-645df4069ae6 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:28.382175 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e54572-2537-4afd-a1b5-645df4069ae6 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:28.386984 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c039e-970a-4310-a56c-e17a49d04329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:28.387646 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c039e-970a-4310-a56c-e17a49d04329 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:28.387938 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c039e-970a-4310-a56c-e17a49d04329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:28.388218 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c039e-970a-4310-a56c-e17a49d04329 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:28.388991 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87c039e-970a-4310-a56c-e17a49d04329 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:27:28.389301 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87c039e-970a-4310-a56c-e17a49d04329 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:28.390980 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87c039e-970a-4310-a56c-e17a49d04329 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:28.404799 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 987/2004] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:27:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:27:28.410036 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479d93d3-371e-49b4-9ade-6a22fedf5c7f None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:28.419253 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479d93d3-371e-49b4-9ade-6a22fedf5c7f None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:28.424125 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f49e71-ead0-4366-b3ef-bc9ffbe5d86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:28.424824 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f49e71-ead0-4366-b3ef-bc9ffbe5d86a admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:28.425112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f49e71-ead0-4366-b3ef-bc9ffbe5d86a admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:28.425390 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f49e71-ead0-4366-b3ef-bc9ffbe5d86a admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:28.426097 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f49e71-ead0-4366-b3ef-bc9ffbe5d86a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:28.427997 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f49e71-ead0-4366-b3ef-bc9ffbe5d86a admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:28.561311 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1018/2005] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:27:28 2022] POST /identity/v3/projects => generated 373 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:27:28.566336 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c039e-970a-4310-a56c-e17a49d04329 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:28.580340 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c039e-970a-4310-a56c-e17a49d04329 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:28.677963 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4fca61-249d-48d1-a5d0-ec4ceb7e8f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:28.679007 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4fca61-249d-48d1-a5d0-ec4ceb7e8f70 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:28.679433 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4fca61-249d-48d1-a5d0-ec4ceb7e8f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:28.680302 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4fca61-249d-48d1-a5d0-ec4ceb7e8f70 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:28.681259 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4fca61-249d-48d1-a5d0-ec4ceb7e8f70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:28.693435 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4fca61-249d-48d1-a5d0-ec4ceb7e8f70 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:28.818415 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 988/2006] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:27:28 2022] POST /identity/v3/users => generated 287 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:27:28.822664 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f49e71-ead0-4366-b3ef-bc9ffbe5d86a admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:28.834093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f49e71-ead0-4366-b3ef-bc9ffbe5d86a admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:28.838696 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d06a0e3-7e4c-4081-b5cb-93f3c1e087ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:28.839374 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d06a0e3-7e4c-4081-b5cb-93f3c1e087ea admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:28.839660 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d06a0e3-7e4c-4081-b5cb-93f3c1e087ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:28.839938 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d06a0e3-7e4c-4081-b5cb-93f3c1e087ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:28.840718 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d06a0e3-7e4c-4081-b5cb-93f3c1e087ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:28.842331 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d06a0e3-7e4c-4081-b5cb-93f3c1e087ea admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:28.844826 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:28.853967 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1019/2007] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:27:28.858164 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4fca61-249d-48d1-a5d0-ec4ceb7e8f70 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:28.869346 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4fca61-249d-48d1-a5d0-ec4ceb7e8f70 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:28.874004 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5293fd04-f3f6-49ee-8b99-290bf6b5d4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:28.874684 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293fd04-f3f6-49ee-8b99-290bf6b5d4be admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:28.888199 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293fd04-f3f6-49ee-8b99-290bf6b5d4be admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:28.888199 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293fd04-f3f6-49ee-8b99-290bf6b5d4be admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:28.888199 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5293fd04-f3f6-49ee-8b99-290bf6b5d4be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:28.888199 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5293fd04-f3f6-49ee-8b99-290bf6b5d4be admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:28.893033 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 989/2008] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:27:28.894234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d06a0e3-7e4c-4081-b5cb-93f3c1e087ea admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:28.904625 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d06a0e3-7e4c-4081-b5cb-93f3c1e087ea admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:28.909399 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82849108-8740-4d7a-843d-727206356420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:28.910309 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82849108-8740-4d7a-843d-727206356420 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:28.910584 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82849108-8740-4d7a-843d-727206356420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:28.910892 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82849108-8740-4d7a-843d-727206356420 admin admin] PATH_INFO: `/v3/projects/77d9cc37ff264420ac0beea96ce16881/users/893cfb9a4cd64a5aaed8418731f68aec/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:28.928283 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82849108-8740-4d7a-843d-727206356420 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77d9cc37ff264420ac0beea96ce16881, user_id=893cfb9a4cd64a5aaed8418731f68aec, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:28.939736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82849108-8740-4d7a-843d-727206356420 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:29.092010 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1020/2009] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:27:28 2022] PUT /identity/v3/projects/77d9cc37ff264420ac0beea96ce16881/users/893cfb9a4cd64a5aaed8418731f68aec/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:27:29.102727 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940703d-08b8-4576-9aee-00232e54637f None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:29.103104 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940703d-08b8-4576-9aee-00232e54637f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:29.103293 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940703d-08b8-4576-9aee-00232e54637f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:29.143864 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-7940703d-08b8-4576-9aee-00232e54637f None None] MFA Rules not processed for user `893cfb9a4cd64a5aaed8418731f68aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:29.145351 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7940703d-08b8-4576-9aee-00232e54637f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:29.216745 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 990/2010] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:27:29.228798 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82849108-8740-4d7a-843d-727206356420 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:29.230561 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-82849108-8740-4d7a-843d-727206356420 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:29.297729 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82849108-8740-4d7a-843d-727206356420 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:29.302192 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831ee44-e48f-48e9-9a06-c53c1344695c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:29.302863 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831ee44-e48f-48e9-9a06-c53c1344695c service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:29.303178 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831ee44-e48f-48e9-9a06-c53c1344695c service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:29.303462 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831ee44-e48f-48e9-9a06-c53c1344695c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:29.312756 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c831ee44-e48f-48e9-9a06-c53c1344695c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:29.314899 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c831ee44-e48f-48e9-9a06-c53c1344695c service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:29.324734 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1021/2011] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:27:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:27:29.353222 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940703d-08b8-4576-9aee-00232e54637f None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:29.404943 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:29.445433 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940703d-08b8-4576-9aee-00232e54637f None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:29.450226 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf238b2-c36a-4353-a94d-444a42da9283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:29.451112 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf238b2-c36a-4353-a94d-444a42da9283 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:29.451405 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf238b2-c36a-4353-a94d-444a42da9283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:29.451691 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf238b2-c36a-4353-a94d-444a42da9283 admin admin] PATH_INFO: `/v3/users/893cfb9a4cd64a5aaed8418731f68aec` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:29.452982 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf238b2-c36a-4353-a94d-444a42da9283 admin admin] RBAC: Authorizing `identity:delete_user(user_id=893cfb9a4cd64a5aaed8418731f68aec)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:29.455334 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf238b2-c36a-4353-a94d-444a42da9283 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:29.557224 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831ee44-e48f-48e9-9a06-c53c1344695c service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:29.558627 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c831ee44-e48f-48e9-9a06-c53c1344695c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:29.581697 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831ee44-e48f-48e9-9a06-c53c1344695c service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:29.586851 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399666d-9d06-43bf-ae61-fc753f3354fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:29.587892 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399666d-9d06-43bf-ae61-fc753f3354fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:29.588277 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399666d-9d06-43bf-ae61-fc753f3354fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:29.588685 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399666d-9d06-43bf-ae61-fc753f3354fe admin admin] PATH_INFO: `/v3/users/832438487d43400d8806215e7a0ae457` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:29.590064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399666d-9d06-43bf-ae61-fc753f3354fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=832438487d43400d8806215e7a0ae457)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:29.600177 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399666d-9d06-43bf-ae61-fc753f3354fe admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:29.703513 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1cf238b2-c36a-4353-a94d-444a42da9283 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '893cfb9a4cd64a5aaed8418731f68aec'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:29.711552 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1cf238b2-c36a-4353-a94d-444a42da9283 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '893cfb9a4cd64a5aaed8418731f68aec'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:29.725102 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1cf238b2-c36a-4353-a94d-444a42da9283 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '893cfb9a4cd64a5aaed8418731f68aec'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:29.725829 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1cf238b2-c36a-4353-a94d-444a42da9283 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '893cfb9a4cd64a5aaed8418731f68aec'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:29.726439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1cf238b2-c36a-4353-a94d-444a42da9283 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '893cfb9a4cd64a5aaed8418731f68aec'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:29.824505 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 991/2012] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:27:29 2022] DELETE /identity/v3/users/893cfb9a4cd64a5aaed8418731f68aec => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:27:29.829836 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af08413-19f7-4d72-ad00-31fab2e24f37 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:29.830263 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af08413-19f7-4d72-ad00-31fab2e24f37 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:29.830712 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af08413-19f7-4d72-ad00-31fab2e24f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:29.847632 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6399666d-9d06-43bf-ae61-fc753f3354fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '832438487d43400d8806215e7a0ae457'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:29.852897 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-1af08413-19f7-4d72-ad00-31fab2e24f37 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:29.854809 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-1af08413-19f7-4d72-ad00-31fab2e24f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:29.856082 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6399666d-9d06-43bf-ae61-fc753f3354fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '832438487d43400d8806215e7a0ae457'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:29.856904 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6399666d-9d06-43bf-ae61-fc753f3354fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '832438487d43400d8806215e7a0ae457'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:29.857576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6399666d-9d06-43bf-ae61-fc753f3354fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '832438487d43400d8806215e7a0ae457'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:29.877107 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6399666d-9d06-43bf-ae61-fc753f3354fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '832438487d43400d8806215e7a0ae457'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:29.952970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 992/2013] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:27:29.973677 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af08413-19f7-4d72-ad00-31fab2e24f37 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:29.974406 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1022/2014] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:27:29 2022] DELETE /identity/v3/users/832438487d43400d8806215e7a0ae457 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:27:29.978328 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-1af08413-19f7-4d72-ad00-31fab2e24f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:29.980667 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f86a66-5999-4a8c-8011-64ed8c6ae0b9 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:29.981142 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f86a66-5999-4a8c-8011-64ed8c6ae0b9 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:29.981489 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f86a66-5999-4a8c-8011-64ed8c6ae0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:30.004570 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-95f86a66-5999-4a8c-8011-64ed8c6ae0b9 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:30.005902 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-95f86a66-5999-4a8c-8011-64ed8c6ae0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:30.055681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:30.062255 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af08413-19f7-4d72-ad00-31fab2e24f37 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:30.082484 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:30.112482 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1023/2015] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:27:30.123305 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f86a66-5999-4a8c-8011-64ed8c6ae0b9 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:30.167127 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24e964e-1d62-4e0c-a9aa-e4cc91250587 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:30.168659 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e964e-1d62-4e0c-a9aa-e4cc91250587 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:30.169061 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e964e-1d62-4e0c-a9aa-e4cc91250587 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:30.169468 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e964e-1d62-4e0c-a9aa-e4cc91250587 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:30.183362 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24e964e-1d62-4e0c-a9aa-e4cc91250587 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:30.185026 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24e964e-1d62-4e0c-a9aa-e4cc91250587 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:30.197193 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 993/2016] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:27:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:27:30.209985 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f86a66-5999-4a8c-8011-64ed8c6ae0b9 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:30.213870 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb894bcb-6067-4825-8c8a-e9ef0e30eb61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:30.214562 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb894bcb-6067-4825-8c8a-e9ef0e30eb61 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:30.214827 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb894bcb-6067-4825-8c8a-e9ef0e30eb61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:30.215106 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb894bcb-6067-4825-8c8a-e9ef0e30eb61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:30.229821 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb894bcb-6067-4825-8c8a-e9ef0e30eb61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:30.231282 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb894bcb-6067-4825-8c8a-e9ef0e30eb61 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:30.242437 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1024/2017] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:27:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:27:31.010608 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24e964e-1d62-4e0c-a9aa-e4cc91250587 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:31.012133 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f24e964e-1d62-4e0c-a9aa-e4cc91250587 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:31.025161 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24e964e-1d62-4e0c-a9aa-e4cc91250587 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:31.030298 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a492f0-f613-4bd9-ac5b-c3a9cd4abfac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:31.031191 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a492f0-f613-4bd9-ac5b-c3a9cd4abfac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:31.031479 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a492f0-f613-4bd9-ac5b-c3a9cd4abfac admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:31.031761 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a492f0-f613-4bd9-ac5b-c3a9cd4abfac admin admin] PATH_INFO: `/v3/users/09791fda27424feda8db22ba0b481e1b` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:31.034193 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a492f0-f613-4bd9-ac5b-c3a9cd4abfac admin admin] RBAC: Authorizing `identity:delete_user(user_id=09791fda27424feda8db22ba0b481e1b)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:31.036345 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a492f0-f613-4bd9-ac5b-c3a9cd4abfac admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:31.107658 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb894bcb-6067-4825-8c8a-e9ef0e30eb61 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:31.120175 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-bb894bcb-6067-4825-8c8a-e9ef0e30eb61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:31.141551 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb894bcb-6067-4825-8c8a-e9ef0e30eb61 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:31.150951 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca0578-c2b9-4df1-88aa-562510de12e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:31.153219 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca0578-c2b9-4df1-88aa-562510de12e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:31.157055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca0578-c2b9-4df1-88aa-562510de12e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:31.157055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca0578-c2b9-4df1-88aa-562510de12e8 admin admin] PATH_INFO: `/v3/projects/77d9cc37ff264420ac0beea96ce16881` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:31.157055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ca0578-c2b9-4df1-88aa-562510de12e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77d9cc37ff264420ac0beea96ce16881)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:31.160906 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ca0578-c2b9-4df1-88aa-562510de12e8 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:31.276717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-01a492f0-f613-4bd9-ac5b-c3a9cd4abfac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09791fda27424feda8db22ba0b481e1b'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:31.294934 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-01a492f0-f613-4bd9-ac5b-c3a9cd4abfac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09791fda27424feda8db22ba0b481e1b'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:31.333947 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-01a492f0-f613-4bd9-ac5b-c3a9cd4abfac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09791fda27424feda8db22ba0b481e1b'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:31.337092 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-01a492f0-f613-4bd9-ac5b-c3a9cd4abfac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09791fda27424feda8db22ba0b481e1b'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:31.337092 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-01a492f0-f613-4bd9-ac5b-c3a9cd4abfac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09791fda27424feda8db22ba0b481e1b'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:31.340522 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-29ca0578-c2b9-4df1-88aa-562510de12e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77d9cc37ff264420ac0beea96ce16881', 'request_id': 'req-29ca0578-c2b9-4df1-88aa-562510de12e8'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:31.353004 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-29ca0578-c2b9-4df1-88aa-562510de12e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77d9cc37ff264420ac0beea96ce16881', 'request_id': 'req-29ca0578-c2b9-4df1-88aa-562510de12e8'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:31.482558 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 994/2018] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:27:31 2022] DELETE /identity/v3/users/09791fda27424feda8db22ba0b481e1b => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:27:31.484103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a492f0-f613-4bd9-ac5b-c3a9cd4abfac admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:31.489457 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-01a492f0-f613-4bd9-ac5b-c3a9cd4abfac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:31.518740 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-29ca0578-c2b9-4df1-88aa-562510de12e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77d9cc37ff264420ac0beea96ce16881', 'request_id': 'req-29ca0578-c2b9-4df1-88aa-562510de12e8'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:31.535160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-29ca0578-c2b9-4df1-88aa-562510de12e8 admin admin] The token cache is being invalidate because project 77d9cc37ff264420ac0beea96ce16881 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:27:31.535465 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-29ca0578-c2b9-4df1-88aa-562510de12e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:31.536120 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-29ca0578-c2b9-4df1-88aa-562510de12e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:31.549883 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1025/2019] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:27:31 2022] DELETE /identity/v3/projects/77d9cc37ff264420ac0beea96ce16881 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:27:31.555799 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ccd4f-acc6-4f12-8a55-37a063eeb78a None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:31.555799 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ccd4f-acc6-4f12-8a55-37a063eeb78a None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:31.556075 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ccd4f-acc6-4f12-8a55-37a063eeb78a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:31.602322 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a492f0-f613-4bd9-ac5b-c3a9cd4abfac admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:31.602713 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-a21ccd4f-acc6-4f12-8a55-37a063eeb78a None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:31.605089 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-a21ccd4f-acc6-4f12-8a55-37a063eeb78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:31.608658 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a7dbf0-8fac-427c-ab72-c9b28ca0ac6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:31.609941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7dbf0-8fac-427c-ab72-c9b28ca0ac6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:31.610385 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7dbf0-8fac-427c-ab72-c9b28ca0ac6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:31.611400 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7dbf0-8fac-427c-ab72-c9b28ca0ac6f admin admin] PATH_INFO: `/v3/projects/93d5b71aa4684e1c8b3b723a45d640df` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:31.612906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a7dbf0-8fac-427c-ab72-c9b28ca0ac6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=93d5b71aa4684e1c8b3b723a45d640df)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:31.615193 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a7dbf0-8fac-427c-ab72-c9b28ca0ac6f admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:31.620591 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1026/2020] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:27:31.620591 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e649dd4-5f7d-400c-a7d5-bcc0d85fa196 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:31.620591 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e649dd4-5f7d-400c-a7d5-bcc0d85fa196 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:31.628925 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e649dd4-5f7d-400c-a7d5-bcc0d85fa196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:31.649969 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:31.665744 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-0e649dd4-5f7d-400c-a7d5-bcc0d85fa196 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:31.666999 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-0e649dd4-5f7d-400c-a7d5-bcc0d85fa196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:31.676188 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1027/2021] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:27:31.688955 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e649dd4-5f7d-400c-a7d5-bcc0d85fa196 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:31.688955 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-0e649dd4-5f7d-400c-a7d5-bcc0d85fa196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:31.748497 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e649dd4-5f7d-400c-a7d5-bcc0d85fa196 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:31.754204 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712216e-5b9a-49b9-8231-09ba8491e194 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:31.755357 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712216e-5b9a-49b9-8231-09ba8491e194 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:31.756062 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712216e-5b9a-49b9-8231-09ba8491e194 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:31.756626 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712216e-5b9a-49b9-8231-09ba8491e194 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:31.776726 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2712216e-5b9a-49b9-8231-09ba8491e194 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:31.778588 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2712216e-5b9a-49b9-8231-09ba8491e194 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:31.790596 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1028/2022] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:27:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:27:31.792315 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e011b087-9d40-4ade-a9b1-a97efff0136f None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:31.792727 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e011b087-9d40-4ade-a9b1-a97efff0136f None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:31.792991 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e011b087-9d40-4ade-a9b1-a97efff0136f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:31.795345 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-24a7dbf0-8fac-427c-ab72-c9b28ca0ac6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93d5b71aa4684e1c8b3b723a45d640df', 'request_id': 'req-24a7dbf0-8fac-427c-ab72-c9b28ca0ac6f'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:31.801934 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:31.807404 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-24a7dbf0-8fac-427c-ab72-c9b28ca0ac6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93d5b71aa4684e1c8b3b723a45d640df', 'request_id': 'req-24a7dbf0-8fac-427c-ab72-c9b28ca0ac6f'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:31.823559 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-e011b087-9d40-4ade-a9b1-a97efff0136f None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:31.825868 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e011b087-9d40-4ade-a9b1-a97efff0136f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:31.878818 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-24a7dbf0-8fac-427c-ab72-c9b28ca0ac6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93d5b71aa4684e1c8b3b723a45d640df', 'request_id': 'req-24a7dbf0-8fac-427c-ab72-c9b28ca0ac6f'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:31.894023 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-24a7dbf0-8fac-427c-ab72-c9b28ca0ac6f admin admin] The token cache is being invalidate because project 93d5b71aa4684e1c8b3b723a45d640df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:27:31.894417 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-24a7dbf0-8fac-427c-ab72-c9b28ca0ac6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:31.895219 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-24a7dbf0-8fac-427c-ab72-c9b28ca0ac6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:31.897721 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 995/2023] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:27:31 2022] DELETE /identity/v3/projects/93d5b71aa4684e1c8b3b723a45d640df => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:27:31.901958 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a7dbf0-8fac-427c-ab72-c9b28ca0ac6f admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:31.904480 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-24a7dbf0-8fac-427c-ab72-c9b28ca0ac6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:31.923597 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1029/2024] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:27:31.927814 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e011b087-9d40-4ade-a9b1-a97efff0136f None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:31.937903 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e011b087-9d40-4ade-a9b1-a97efff0136f None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:31.943015 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb6e53e-0932-437b-832d-8f9ea59ad2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:31.943714 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb6e53e-0932-437b-832d-8f9ea59ad2f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:31.944033 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb6e53e-0932-437b-832d-8f9ea59ad2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:31.944383 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb6e53e-0932-437b-832d-8f9ea59ad2f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:31.945174 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb6e53e-0932-437b-832d-8f9ea59ad2f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:27:31.945493 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb6e53e-0932-437b-832d-8f9ea59ad2f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:31.947211 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb6e53e-0932-437b-832d-8f9ea59ad2f1 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:31.962751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1030/2025] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:27:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:27:31.966427 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb6e53e-0932-437b-832d-8f9ea59ad2f1 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:31.979064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb6e53e-0932-437b-832d-8f9ea59ad2f1 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:31.984810 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c4a93-a383-496e-b5e9-1505dcdb35bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:31.986091 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c4a93-a383-496e-b5e9-1505dcdb35bf admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:31.986396 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c4a93-a383-496e-b5e9-1505dcdb35bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:31.986681 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c4a93-a383-496e-b5e9-1505dcdb35bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:31.987495 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7c4a93-a383-496e-b5e9-1505dcdb35bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:31.989521 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7c4a93-a383-496e-b5e9-1505dcdb35bf admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:31.998191 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a7dbf0-8fac-427c-ab72-c9b28ca0ac6f admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:32.008354 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421cfdf9-e065-42fa-8eae-f79a51bc7fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:32.009292 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421cfdf9-e065-42fa-8eae-f79a51bc7fcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:32.009583 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421cfdf9-e065-42fa-8eae-f79a51bc7fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:32.009958 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421cfdf9-e065-42fa-8eae-f79a51bc7fcd admin admin] PATH_INFO: `/v3/users/7678ca998ea44013ba079a9f4daed196` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:32.011348 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421cfdf9-e065-42fa-8eae-f79a51bc7fcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=7678ca998ea44013ba079a9f4daed196)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:32.013506 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421cfdf9-e065-42fa-8eae-f79a51bc7fcd admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:32.153658 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1031/2026] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:27:31 2022] POST /identity/v3/projects => generated 373 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:27:32.155338 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c4a93-a383-496e-b5e9-1505dcdb35bf admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:32.242971 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c4a93-a383-496e-b5e9-1505dcdb35bf admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:32.248515 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4bac4a-a0b8-46e5-a34d-35d8600180de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:32.249143 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bac4a-a0b8-46e5-a34d-35d8600180de admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:32.249469 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bac4a-a0b8-46e5-a34d-35d8600180de admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:32.249751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bac4a-a0b8-46e5-a34d-35d8600180de admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:32.250451 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4bac4a-a0b8-46e5-a34d-35d8600180de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:32.252503 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4bac4a-a0b8-46e5-a34d-35d8600180de admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:32.279307 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-421cfdf9-e065-42fa-8eae-f79a51bc7fcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7678ca998ea44013ba079a9f4daed196'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:32.287711 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-421cfdf9-e065-42fa-8eae-f79a51bc7fcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7678ca998ea44013ba079a9f4daed196'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:32.299929 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-421cfdf9-e065-42fa-8eae-f79a51bc7fcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7678ca998ea44013ba079a9f4daed196'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:32.300704 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-421cfdf9-e065-42fa-8eae-f79a51bc7fcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7678ca998ea44013ba079a9f4daed196'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:32.301367 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-421cfdf9-e065-42fa-8eae-f79a51bc7fcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7678ca998ea44013ba079a9f4daed196'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:32.356979 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1032/2027] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:27:32 2022] POST /identity/v3/users => generated 287 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:27:32.356979 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4bac4a-a0b8-46e5-a34d-35d8600180de admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:32.359814 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-2e4bac4a-a0b8-46e5-a34d-35d8600180de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:32.373267 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4bac4a-a0b8-46e5-a34d-35d8600180de admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:32.380018 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f033d29-7e3b-4f0f-b4ff-28e1b0805289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:32.391427 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f033d29-7e3b-4f0f-b4ff-28e1b0805289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:32.391427 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f033d29-7e3b-4f0f-b4ff-28e1b0805289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:32.391427 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f033d29-7e3b-4f0f-b4ff-28e1b0805289 admin admin] PATH_INFO: `/v3/projects/257b82c51e534cd98b484351a11a17da` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:32.391427 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f033d29-7e3b-4f0f-b4ff-28e1b0805289 admin admin] RBAC: Authorizing `identity:delete_project(project_id=257b82c51e534cd98b484351a11a17da)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:32.391427 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f033d29-7e3b-4f0f-b4ff-28e1b0805289 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:32.427382 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 996/2028] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:27:31 2022] DELETE /identity/v3/users/7678ca998ea44013ba079a9f4daed196 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:27:32.431580 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421cfdf9-e065-42fa-8eae-f79a51bc7fcd admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:32.432761 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-421cfdf9-e065-42fa-8eae-f79a51bc7fcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:32.533998 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421cfdf9-e065-42fa-8eae-f79a51bc7fcd admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:32.542325 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-5f033d29-7e3b-4f0f-b4ff-28e1b0805289 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '257b82c51e534cd98b484351a11a17da', 'request_id': 'req-5f033d29-7e3b-4f0f-b4ff-28e1b0805289'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:32.551559 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-5f033d29-7e3b-4f0f-b4ff-28e1b0805289 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '257b82c51e534cd98b484351a11a17da', 'request_id': 'req-5f033d29-7e3b-4f0f-b4ff-28e1b0805289'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:32.614988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b8d821-8660-4423-8e71-345c2ac67699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:32.615743 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8d821-8660-4423-8e71-345c2ac67699 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:32.616032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8d821-8660-4423-8e71-345c2ac67699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:32.616310 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8d821-8660-4423-8e71-345c2ac67699 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:32.617099 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b8d821-8660-4423-8e71-345c2ac67699 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:32.618791 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b8d821-8660-4423-8e71-345c2ac67699 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:32.631426 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 997/2029] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:27:32 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:27:32.635605 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b8d821-8660-4423-8e71-345c2ac67699 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:32.647212 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b8d821-8660-4423-8e71-345c2ac67699 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:32.653215 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfadea87-549a-4525-8ea0-f6557519a814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:32.655001 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfadea87-549a-4525-8ea0-f6557519a814 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:32.655268 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfadea87-549a-4525-8ea0-f6557519a814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:32.655550 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfadea87-549a-4525-8ea0-f6557519a814 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:32.656311 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfadea87-549a-4525-8ea0-f6557519a814 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:32.658252 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfadea87-549a-4525-8ea0-f6557519a814 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:32.665950 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-5f033d29-7e3b-4f0f-b4ff-28e1b0805289 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '257b82c51e534cd98b484351a11a17da', 'request_id': 'req-5f033d29-7e3b-4f0f-b4ff-28e1b0805289'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:32.672491 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 998/2030] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:27:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 06:27:32.675191 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfadea87-549a-4525-8ea0-f6557519a814 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:32.681471 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-5f033d29-7e3b-4f0f-b4ff-28e1b0805289 admin admin] The token cache is being invalidate because project 257b82c51e534cd98b484351a11a17da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:27:32.681815 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-5f033d29-7e3b-4f0f-b4ff-28e1b0805289 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:32.682487 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-5f033d29-7e3b-4f0f-b4ff-28e1b0805289 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:32.687807 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1033/2031] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:27:32 2022] DELETE /identity/v3/projects/257b82c51e534cd98b484351a11a17da => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:27:32.690526 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026628d-0bf3-4db0-9468-44abde4f40a2 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:32.691256 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026628d-0bf3-4db0-9468-44abde4f40a2 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:32.691256 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026628d-0bf3-4db0-9468-44abde4f40a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:32.698840 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfadea87-549a-4525-8ea0-f6557519a814 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:32.707238 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e80cb-685a-48ab-a002-c30854820f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:32.708466 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e80cb-685a-48ab-a002-c30854820f0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:32.709050 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e80cb-685a-48ab-a002-c30854820f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:32.709050 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e80cb-685a-48ab-a002-c30854820f0a admin admin] PATH_INFO: `/v3/projects/0c2f0aa6575940e7b01c039449ad0659/users/ba7eb26d3cf04471ac34ececfa569b43/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:32.716494 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-2026628d-0bf3-4db0-9468-44abde4f40a2 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:32.718079 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-2026628d-0bf3-4db0-9468-44abde4f40a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:32.736598 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1034/2032] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:27:32.737996 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8048de-55a0-46ff-afd8-021f8fbb24c8 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:32.738339 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8048de-55a0-46ff-afd8-021f8fbb24c8 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:32.738628 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8048de-55a0-46ff-afd8-021f8fbb24c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:32.740827 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e80cb-685a-48ab-a002-c30854820f0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c2f0aa6575940e7b01c039449ad0659, user_id=ba7eb26d3cf04471ac34ececfa569b43, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:32.749555 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e80cb-685a-48ab-a002-c30854820f0a admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:32.773131 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-9b8048de-55a0-46ff-afd8-021f8fbb24c8 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:32.774425 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-9b8048de-55a0-46ff-afd8-021f8fbb24c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:32.800512 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1035/2033] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:27:32.805020 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8048de-55a0-46ff-afd8-021f8fbb24c8 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:32.826915 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 999/2034] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:27:32 2022] PUT /identity/v3/projects/0c2f0aa6575940e7b01c039449ad0659/users/ba7eb26d3cf04471ac34ececfa569b43/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:27:32.827148 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8048de-55a0-46ff-afd8-021f8fbb24c8 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:32.836478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9beb1-f3fa-46ae-b1e8-cdbb7fe808eb None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:32.836775 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9beb1-f3fa-46ae-b1e8-cdbb7fe808eb None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:32.837048 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9beb1-f3fa-46ae-b1e8-cdbb7fe808eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:32.867847 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-ecc9beb1-f3fa-46ae-b1e8-cdbb7fe808eb None None] MFA Rules not processed for user `ba7eb26d3cf04471ac34ececfa569b43`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:32.870339 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ecc9beb1-f3fa-46ae-b1e8-cdbb7fe808eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:32.950530 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965532f-0836-4dcc-a487-5d7ea4e15454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:32.951241 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965532f-0836-4dcc-a487-5d7ea4e15454 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:32.951526 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965532f-0836-4dcc-a487-5d7ea4e15454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:32.951800 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965532f-0836-4dcc-a487-5d7ea4e15454 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:32.952573 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6965532f-0836-4dcc-a487-5d7ea4e15454 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:27:32.952881 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6965532f-0836-4dcc-a487-5d7ea4e15454 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:32.956213 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6965532f-0836-4dcc-a487-5d7ea4e15454 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:32.971109 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1000/2035] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:27:32.976782 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1036/2036] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:27:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:27:32.982931 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc9beb1-f3fa-46ae-b1e8-cdbb7fe808eb None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:32.983240 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965532f-0836-4dcc-a487-5d7ea4e15454 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:32.984971 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-6965532f-0836-4dcc-a487-5d7ea4e15454 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:32.995285 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc9beb1-f3fa-46ae-b1e8-cdbb7fe808eb None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:33.000100 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e436122c-b830-432a-900f-f220e0fee062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:33.000818 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436122c-b830-432a-900f-f220e0fee062 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:33.001212 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436122c-b830-432a-900f-f220e0fee062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:33.001502 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436122c-b830-432a-900f-f220e0fee062 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:33.002283 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e436122c-b830-432a-900f-f220e0fee062 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:33.004681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e436122c-b830-432a-900f-f220e0fee062 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:33.048638 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965532f-0836-4dcc-a487-5d7ea4e15454 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:33.052433 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2bc72b-25ec-46f0-978f-0812719e94c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:33.053250 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bc72b-25ec-46f0-978f-0812719e94c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:33.053497 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bc72b-25ec-46f0-978f-0812719e94c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:33.053783 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bc72b-25ec-46f0-978f-0812719e94c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:33.063415 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2bc72b-25ec-46f0-978f-0812719e94c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:33.065042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2bc72b-25ec-46f0-978f-0812719e94c5 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:33.074892 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1037/2037] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:27:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 06:27:33.075959 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2bc72b-25ec-46f0-978f-0812719e94c5 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:33.077309 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-fa2bc72b-25ec-46f0-978f-0812719e94c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:33.094015 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2bc72b-25ec-46f0-978f-0812719e94c5 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:33.101376 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1001/2038] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:27:32 2022] POST /identity/v3/projects => generated 365 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:27:33.101923 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bfb73d-6d0c-4332-9bf0-f56ab59e3f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:33.102178 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bfb73d-6d0c-4332-9bf0-f56ab59e3f26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:33.102514 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bfb73d-6d0c-4332-9bf0-f56ab59e3f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:33.102965 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bfb73d-6d0c-4332-9bf0-f56ab59e3f26 admin admin] PATH_INFO: `/v3/projects/f93d922736d04c9caefeaeedb77fbb44` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:33.105195 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bfb73d-6d0c-4332-9bf0-f56ab59e3f26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f93d922736d04c9caefeaeedb77fbb44)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:33.105889 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e436122c-b830-432a-900f-f220e0fee062 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:33.108464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bfb73d-6d0c-4332-9bf0-f56ab59e3f26 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:33.184198 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e436122c-b830-432a-900f-f220e0fee062 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:33.189814 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217bed6c-aee7-43f9-9956-4fb6c5babc06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:33.190635 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217bed6c-aee7-43f9-9956-4fb6c5babc06 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:33.190930 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217bed6c-aee7-43f9-9956-4fb6c5babc06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:33.191209 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217bed6c-aee7-43f9-9956-4fb6c5babc06 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:33.191919 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217bed6c-aee7-43f9-9956-4fb6c5babc06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:33.193889 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217bed6c-aee7-43f9-9956-4fb6c5babc06 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:33.246862 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d6bfb73d-6d0c-4332-9bf0-f56ab59e3f26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f93d922736d04c9caefeaeedb77fbb44', 'request_id': 'req-d6bfb73d-6d0c-4332-9bf0-f56ab59e3f26'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:33.253332 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d6bfb73d-6d0c-4332-9bf0-f56ab59e3f26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f93d922736d04c9caefeaeedb77fbb44', 'request_id': 'req-d6bfb73d-6d0c-4332-9bf0-f56ab59e3f26'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:33.301242 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1002/2039] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:27:33 2022] POST /identity/v3/users => generated 283 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:27:33.305148 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217bed6c-aee7-43f9-9956-4fb6c5babc06 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:33.314956 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217bed6c-aee7-43f9-9956-4fb6c5babc06 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:33.324633 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d6bfb73d-6d0c-4332-9bf0-f56ab59e3f26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f93d922736d04c9caefeaeedb77fbb44', 'request_id': 'req-d6bfb73d-6d0c-4332-9bf0-f56ab59e3f26'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:33.343191 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d6bfb73d-6d0c-4332-9bf0-f56ab59e3f26 admin admin] The token cache is being invalidate because project f93d922736d04c9caefeaeedb77fbb44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:27:33.343490 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d6bfb73d-6d0c-4332-9bf0-f56ab59e3f26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:33.344231 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d6bfb73d-6d0c-4332-9bf0-f56ab59e3f26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:33.347592 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1038/2040] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:27:33 2022] DELETE /identity/v3/projects/f93d922736d04c9caefeaeedb77fbb44 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:27:33.352121 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bfb73d-6d0c-4332-9bf0-f56ab59e3f26 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:33.353657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d6bfb73d-6d0c-4332-9bf0-f56ab59e3f26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:33.383129 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d403e95-583a-4060-b02e-30d7ace31032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:33.383828 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d403e95-583a-4060-b02e-30d7ace31032 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:33.384238 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d403e95-583a-4060-b02e-30d7ace31032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:33.384494 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d403e95-583a-4060-b02e-30d7ace31032 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:33.385260 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d403e95-583a-4060-b02e-30d7ace31032 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:33.386888 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d403e95-583a-4060-b02e-30d7ace31032 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:33.398775 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1003/2041] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 06:27:33.403318 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d403e95-583a-4060-b02e-30d7ace31032 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:33.405260 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-2d403e95-583a-4060-b02e-30d7ace31032 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:33.419072 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d403e95-583a-4060-b02e-30d7ace31032 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:33.425135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6fcc70-0466-4d56-aeab-eb81cdde6be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:33.425732 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bfb73d-6d0c-4332-9bf0-f56ab59e3f26 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:33.425843 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6fcc70-0466-4d56-aeab-eb81cdde6be0 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:33.426080 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6fcc70-0466-4d56-aeab-eb81cdde6be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:33.426369 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6fcc70-0466-4d56-aeab-eb81cdde6be0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:33.427102 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6fcc70-0466-4d56-aeab-eb81cdde6be0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:33.428734 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6fcc70-0466-4d56-aeab-eb81cdde6be0 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:33.431403 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9ee30-9344-4885-b0e0-3eb6bbf1d762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:33.432438 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:33.432590 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9ee30-9344-4885-b0e0-3eb6bbf1d762 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:33.432987 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9ee30-9344-4885-b0e0-3eb6bbf1d762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:33.433275 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9ee30-9344-4885-b0e0-3eb6bbf1d762 admin admin] PATH_INFO: `/v3/users/b8984365b1db4f06bf0c346e865e4796` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:33.434573 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e9ee30-9344-4885-b0e0-3eb6bbf1d762 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8984365b1db4f06bf0c346e865e4796)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:33.436620 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e9ee30-9344-4885-b0e0-3eb6bbf1d762 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:33.443031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1004/2042] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 06:27:33.445591 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6fcc70-0466-4d56-aeab-eb81cdde6be0 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:33.455728 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6fcc70-0466-4d56-aeab-eb81cdde6be0 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:33.460512 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee4c18-58e6-497e-849a-3e3eb9e500cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:33.461413 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee4c18-58e6-497e-849a-3e3eb9e500cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:33.461698 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee4c18-58e6-497e-849a-3e3eb9e500cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:33.461974 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee4c18-58e6-497e-849a-3e3eb9e500cd admin admin] PATH_INFO: `/v3/projects/575ddb99e9eb4fcf8b0c8cc1ac574616/users/519f82e56dec4aed9bafa7dea6a436fb/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:33.479063 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee4c18-58e6-497e-849a-3e3eb9e500cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=575ddb99e9eb4fcf8b0c8cc1ac574616, user_id=519f82e56dec4aed9bafa7dea6a436fb, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:33.481975 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee4c18-58e6-497e-849a-3e3eb9e500cd admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:33.596217 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1005/2043] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:27:33 2022] PUT /identity/v3/projects/575ddb99e9eb4fcf8b0c8cc1ac574616/users/519f82e56dec4aed9bafa7dea6a436fb/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:27:33.608945 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e40c3-848e-43e6-87f4-486ef4d89ff8 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:33.609434 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e40c3-848e-43e6-87f4-486ef4d89ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:33.609968 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e40c3-848e-43e6-87f4-486ef4d89ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:33.641277 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-8e4e40c3-848e-43e6-87f4-486ef4d89ff8 None None] MFA Rules not processed for user `519f82e56dec4aed9bafa7dea6a436fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:33.643500 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8e4e40c3-848e-43e6-87f4-486ef4d89ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:33.664193 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-50e9ee30-9344-4885-b0e0-3eb6bbf1d762 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8984365b1db4f06bf0c346e865e4796'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:33.671174 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-50e9ee30-9344-4885-b0e0-3eb6bbf1d762 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8984365b1db4f06bf0c346e865e4796'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:33.671875 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-50e9ee30-9344-4885-b0e0-3eb6bbf1d762 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8984365b1db4f06bf0c346e865e4796'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:33.672495 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-50e9ee30-9344-4885-b0e0-3eb6bbf1d762 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8984365b1db4f06bf0c346e865e4796'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:33.685530 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-50e9ee30-9344-4885-b0e0-3eb6bbf1d762 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8984365b1db4f06bf0c346e865e4796'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:33.710265 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1006/2044] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:33 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:27:33.723172 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e40c3-848e-43e6-87f4-486ef4d89ff8 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:33.724622 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8e4e40c3-848e-43e6-87f4-486ef4d89ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:33.771296 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1039/2045] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:27:33 2022] DELETE /identity/v3/users/b8984365b1db4f06bf0c346e865e4796 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:27:33.792276 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:33.802086 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e40c3-848e-43e6-87f4-486ef4d89ff8 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:33.897595 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00264de2-ca28-4490-94dd-ffcc7af74963 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:33.898353 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00264de2-ca28-4490-94dd-ffcc7af74963 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:33.898646 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00264de2-ca28-4490-94dd-ffcc7af74963 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:33.898980 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00264de2-ca28-4490-94dd-ffcc7af74963 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:33.972869 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00264de2-ca28-4490-94dd-ffcc7af74963 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:33.973789 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00264de2-ca28-4490-94dd-ffcc7af74963 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:33.989651 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1007/2046] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:27:33 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 06:27:34.394916 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9ee30-9344-4885-b0e0-3eb6bbf1d762 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:34.402360 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-50e9ee30-9344-4885-b0e0-3eb6bbf1d762 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:34.465793 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9ee30-9344-4885-b0e0-3eb6bbf1d762 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:34.471042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560b920-aed9-4e72-bc12-65d18804d123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:34.472042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560b920-aed9-4e72-bc12-65d18804d123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:34.472331 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560b920-aed9-4e72-bc12-65d18804d123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:34.472649 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560b920-aed9-4e72-bc12-65d18804d123 admin admin] PATH_INFO: `/v3/projects/f6478d3d7bf04fd988ef5d7595108080` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:34.473898 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a560b920-aed9-4e72-bc12-65d18804d123 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6478d3d7bf04fd988ef5d7595108080)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:34.475809 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a560b920-aed9-4e72-bc12-65d18804d123 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:34.620499 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-a560b920-aed9-4e72-bc12-65d18804d123 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6478d3d7bf04fd988ef5d7595108080', 'request_id': 'req-a560b920-aed9-4e72-bc12-65d18804d123'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:34.624097 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-a560b920-aed9-4e72-bc12-65d18804d123 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6478d3d7bf04fd988ef5d7595108080', 'request_id': 'req-a560b920-aed9-4e72-bc12-65d18804d123'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:34.706392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-a560b920-aed9-4e72-bc12-65d18804d123 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6478d3d7bf04fd988ef5d7595108080', 'request_id': 'req-a560b920-aed9-4e72-bc12-65d18804d123'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:34.718008 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-a560b920-aed9-4e72-bc12-65d18804d123 admin admin] The token cache is being invalidate because project f6478d3d7bf04fd988ef5d7595108080 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:27:34.718304 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-a560b920-aed9-4e72-bc12-65d18804d123 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:34.719038 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-a560b920-aed9-4e72-bc12-65d18804d123 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:34.721711 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1040/2047] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:27:34 2022] DELETE /identity/v3/projects/f6478d3d7bf04fd988ef5d7595108080 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:27:34.726698 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef506ff-b8b2-46bf-89d3-d8deacf064fc None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:34.727397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef506ff-b8b2-46bf-89d3-d8deacf064fc None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:34.727397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef506ff-b8b2-46bf-89d3-d8deacf064fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:34.747391 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-5ef506ff-b8b2-46bf-89d3-d8deacf064fc None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:34.748668 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-5ef506ff-b8b2-46bf-89d3-d8deacf064fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:34.825546 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1008/2048] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:27:34.825651 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443d038-6166-4376-854f-45b14241c779 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:34.825651 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443d038-6166-4376-854f-45b14241c779 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:34.825651 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443d038-6166-4376-854f-45b14241c779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:34.843489 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-9443d038-6166-4376-854f-45b14241c779 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:34.848294 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-9443d038-6166-4376-854f-45b14241c779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:34.854134 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1041/2049] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:27:34.858397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef506ff-b8b2-46bf-89d3-d8deacf064fc None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:34.866779 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:34.870640 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef506ff-b8b2-46bf-89d3-d8deacf064fc None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:34.879909 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf9cf70-ca97-4e26-a627-7b27d890ecef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:34.879909 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9cf70-ca97-4e26-a627-7b27d890ecef admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:34.879909 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9cf70-ca97-4e26-a627-7b27d890ecef admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:34.879909 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9cf70-ca97-4e26-a627-7b27d890ecef admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:34.880296 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf9cf70-ca97-4e26-a627-7b27d890ecef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:27:34.880296 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf9cf70-ca97-4e26-a627-7b27d890ecef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:34.880296 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf9cf70-ca97-4e26-a627-7b27d890ecef admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:34.891537 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1009/2050] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:27:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:27:34.901093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443d038-6166-4376-854f-45b14241c779 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:34.906341 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443d038-6166-4376-854f-45b14241c779 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:34.913535 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348beb5-570b-41ec-bfc5-269cce6b17ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:34.914017 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348beb5-570b-41ec-bfc5-269cce6b17ef admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:34.914299 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348beb5-570b-41ec-bfc5-269cce6b17ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:34.914582 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348beb5-570b-41ec-bfc5-269cce6b17ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:34.915316 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d348beb5-570b-41ec-bfc5-269cce6b17ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:34.917452 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d348beb5-570b-41ec-bfc5-269cce6b17ef admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:35.015982 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1042/2051] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:27:34 2022] POST /identity/v3/projects => generated 367 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:27:35.021185 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf9cf70-ca97-4e26-a627-7b27d890ecef admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:35.032988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf9cf70-ca97-4e26-a627-7b27d890ecef admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:35.093312 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f366ba-4425-4510-be37-3bafcfd2e265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:35.094018 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f366ba-4425-4510-be37-3bafcfd2e265 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:35.094291 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f366ba-4425-4510-be37-3bafcfd2e265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:35.094572 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f366ba-4425-4510-be37-3bafcfd2e265 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:35.096247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f366ba-4425-4510-be37-3bafcfd2e265 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:35.098439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f366ba-4425-4510-be37-3bafcfd2e265 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:35.131146 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:35.194114 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1010/2052] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:27:35 2022] POST /identity/v3/users => generated 284 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:27:35.198380 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348beb5-570b-41ec-bfc5-269cce6b17ef admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:35.209460 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348beb5-570b-41ec-bfc5-269cce6b17ef admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:35.214543 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041741b-0a64-4849-838b-ac05525e64f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:35.215212 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041741b-0a64-4849-838b-ac05525e64f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:35.215498 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041741b-0a64-4849-838b-ac05525e64f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:35.215780 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041741b-0a64-4849-838b-ac05525e64f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:35.216559 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4041741b-0a64-4849-838b-ac05525e64f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:35.218174 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4041741b-0a64-4849-838b-ac05525e64f4 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:35.231668 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1043/2053] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:27:35.243075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f366ba-4425-4510-be37-3bafcfd2e265 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:35.254968 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f366ba-4425-4510-be37-3bafcfd2e265 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:35.260371 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078b55a-2056-4f5f-ae54-96bb7d0eaafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:35.261068 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078b55a-2056-4f5f-ae54-96bb7d0eaafe admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:35.261352 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078b55a-2056-4f5f-ae54-96bb7d0eaafe admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:35.261631 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078b55a-2056-4f5f-ae54-96bb7d0eaafe admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:35.268084 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3078b55a-2056-4f5f-ae54-96bb7d0eaafe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:35.269814 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3078b55a-2056-4f5f-ae54-96bb7d0eaafe admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:35.285326 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1011/2054] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:27:35.289761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041741b-0a64-4849-838b-ac05525e64f4 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:35.302736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041741b-0a64-4849-838b-ac05525e64f4 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:35.307672 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe2d970-a73d-4f1d-8629-d698cac6046f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:35.308697 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2d970-a73d-4f1d-8629-d698cac6046f admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:35.309063 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2d970-a73d-4f1d-8629-d698cac6046f admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:35.309307 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2d970-a73d-4f1d-8629-d698cac6046f admin admin] PATH_INFO: `/v3/projects/69f190aaf4b84ebeb71646e88ba352d6/users/8a19ecad5df944419a5aa3652921f2a9/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:35.325861 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe2d970-a73d-4f1d-8629-d698cac6046f admin admin] RBAC: Authorizing `identity:create_grant(project_id=69f190aaf4b84ebeb71646e88ba352d6, user_id=8a19ecad5df944419a5aa3652921f2a9, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:35.328592 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe2d970-a73d-4f1d-8629-d698cac6046f admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:35.535008 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1044/2055] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:27:35 2022] PUT /identity/v3/projects/69f190aaf4b84ebeb71646e88ba352d6/users/8a19ecad5df944419a5aa3652921f2a9/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:27:35.545458 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83691c-ac9d-421b-9140-94374921ba10 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:35.550798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83691c-ac9d-421b-9140-94374921ba10 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:35.550798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83691c-ac9d-421b-9140-94374921ba10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:35.573653 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-6a83691c-ac9d-421b-9140-94374921ba10 None None] MFA Rules not processed for user `8a19ecad5df944419a5aa3652921f2a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:35.574848 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-6a83691c-ac9d-421b-9140-94374921ba10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:35.597885 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:35.648177 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1012/2056] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:35 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:27:35.655835 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe2d970-a73d-4f1d-8629-d698cac6046f admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:35.658131 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-dbe2d970-a73d-4f1d-8629-d698cac6046f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:35.738682 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe2d970-a73d-4f1d-8629-d698cac6046f admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:35.742517 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc68e144-ae43-4dfb-b59b-84fdf64a3b44 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1193330cb5644765a7abb77738929ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:35.743178 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68e144-ae43-4dfb-b59b-84fdf64a3b44 service glance] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:35.743462 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68e144-ae43-4dfb-b59b-84fdf64a3b44 service glance] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:35.743984 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68e144-ae43-4dfb-b59b-84fdf64a3b44 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:35.756196 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc68e144-ae43-4dfb-b59b-84fdf64a3b44 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:35.757708 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc68e144-ae43-4dfb-b59b-84fdf64a3b44 service glance] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:35.768248 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1045/2057] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:27:35 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:27:35.843299 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:35.874137 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:39.177061 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9bb72-7a28-4d7e-921e-b841597cd9b3 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:39.177714 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9bb72-7a28-4d7e-921e-b841597cd9b3 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:39.177714 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9bb72-7a28-4d7e-921e-b841597cd9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:39.203338 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-6dc9bb72-7a28-4d7e-921e-b841597cd9b3 None None] MFA Rules not processed for user `ba7eb26d3cf04471ac34ececfa569b43`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:39.206165 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-6dc9bb72-7a28-4d7e-921e-b841597cd9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:39.269561 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:39.281521 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1013/2058] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:27:45.205666 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc68e144-ae43-4dfb-b59b-84fdf64a3b44 service glance] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:45.207290 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-fc68e144-ae43-4dfb-b59b-84fdf64a3b44 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:45.267305 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:45.348399 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc68e144-ae43-4dfb-b59b-84fdf64a3b44 service glance] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:45.356792 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e72a22-9dd6-4e64-b4b8-8a7b64ef2292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:45.357780 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e72a22-9dd6-4e64-b4b8-8a7b64ef2292 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:45.358161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e72a22-9dd6-4e64-b4b8-8a7b64ef2292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:45.358371 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e72a22-9dd6-4e64-b4b8-8a7b64ef2292 admin admin] PATH_INFO: `/v3/users/9c8cc43dac064ef4a3da462459a20d50` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:45.359622 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e72a22-9dd6-4e64-b4b8-8a7b64ef2292 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c8cc43dac064ef4a3da462459a20d50)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:45.362814 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e72a22-9dd6-4e64-b4b8-8a7b64ef2292 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:45.575913 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-90e72a22-9dd6-4e64-b4b8-8a7b64ef2292 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c8cc43dac064ef4a3da462459a20d50'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:45.595192 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-90e72a22-9dd6-4e64-b4b8-8a7b64ef2292 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c8cc43dac064ef4a3da462459a20d50'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:45.596007 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-90e72a22-9dd6-4e64-b4b8-8a7b64ef2292 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c8cc43dac064ef4a3da462459a20d50'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:45.598763 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-90e72a22-9dd6-4e64-b4b8-8a7b64ef2292 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c8cc43dac064ef4a3da462459a20d50'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:45.630975 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-90e72a22-9dd6-4e64-b4b8-8a7b64ef2292 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c8cc43dac064ef4a3da462459a20d50'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:45.727161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1046/2059] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:27:45 2022] DELETE /identity/v3/users/9c8cc43dac064ef4a3da462459a20d50 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:27:45.731731 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d41e27-6e8e-4676-85c5-dfc8e4191e18 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:45.732369 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d41e27-6e8e-4676-85c5-dfc8e4191e18 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:45.732666 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d41e27-6e8e-4676-85c5-dfc8e4191e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:45.769560 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-46d41e27-6e8e-4676-85c5-dfc8e4191e18 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:45.771219 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-46d41e27-6e8e-4676-85c5-dfc8e4191e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:45.871088 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1014/2060] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:27:45.927479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e72a22-9dd6-4e64-b4b8-8a7b64ef2292 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:45.930181 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-90e72a22-9dd6-4e64-b4b8-8a7b64ef2292 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:46.038702 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e72a22-9dd6-4e64-b4b8-8a7b64ef2292 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:46.042593 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39d773-3e56-4895-baa6-d1bdaf6d3485 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:46.043504 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39d773-3e56-4895-baa6-d1bdaf6d3485 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:46.043793 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39d773-3e56-4895-baa6-d1bdaf6d3485 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:46.044078 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39d773-3e56-4895-baa6-d1bdaf6d3485 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:46.054220 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc39d773-3e56-4895-baa6-d1bdaf6d3485 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:46.055706 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc39d773-3e56-4895-baa6-d1bdaf6d3485 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:46.066970 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1047/2061] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:27:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:27:46.940876 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d41e27-6e8e-4676-85c5-dfc8e4191e18 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:46.946537 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-46d41e27-6e8e-4676-85c5-dfc8e4191e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:46.959502 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d41e27-6e8e-4676-85c5-dfc8e4191e18 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:46.964496 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382280c6-e0ae-49eb-9c6e-4b561595e3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:46.965576 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382280c6-e0ae-49eb-9c6e-4b561595e3fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:46.965931 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382280c6-e0ae-49eb-9c6e-4b561595e3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:46.966414 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382280c6-e0ae-49eb-9c6e-4b561595e3fb admin admin] PATH_INFO: `/v3/projects/fcc46a296c5b4bd79fed7a261b121aa2` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:46.967903 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382280c6-e0ae-49eb-9c6e-4b561595e3fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcc46a296c5b4bd79fed7a261b121aa2)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:46.970386 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382280c6-e0ae-49eb-9c6e-4b561595e3fb admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:47.126640 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-382280c6-e0ae-49eb-9c6e-4b561595e3fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcc46a296c5b4bd79fed7a261b121aa2', 'request_id': 'req-382280c6-e0ae-49eb-9c6e-4b561595e3fb'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:47.136132 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-382280c6-e0ae-49eb-9c6e-4b561595e3fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcc46a296c5b4bd79fed7a261b121aa2', 'request_id': 'req-382280c6-e0ae-49eb-9c6e-4b561595e3fb'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:47.213686 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-382280c6-e0ae-49eb-9c6e-4b561595e3fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcc46a296c5b4bd79fed7a261b121aa2', 'request_id': 'req-382280c6-e0ae-49eb-9c6e-4b561595e3fb'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:47.236193 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-382280c6-e0ae-49eb-9c6e-4b561595e3fb admin admin] The token cache is being invalidate because project fcc46a296c5b4bd79fed7a261b121aa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:27:47.236752 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-382280c6-e0ae-49eb-9c6e-4b561595e3fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:47.237860 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-382280c6-e0ae-49eb-9c6e-4b561595e3fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:47.240588 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1015/2062] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:27:46 2022] DELETE /identity/v3/projects/fcc46a296c5b4bd79fed7a261b121aa2 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:27:47.245708 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39d773-3e56-4895-baa6-d1bdaf6d3485 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:47.264592 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39d773-3e56-4895-baa6-d1bdaf6d3485 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:47.320246 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:47.339651 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71912ec1-837b-4731-a675-e856266ef426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:47.340766 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71912ec1-837b-4731-a675-e856266ef426 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:47.341055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71912ec1-837b-4731-a675-e856266ef426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:47.341341 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71912ec1-837b-4731-a675-e856266ef426 admin admin] PATH_INFO: `/v3/users/83f9f740cdd9480da49b5a6a07552b4f` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:47.343437 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71912ec1-837b-4731-a675-e856266ef426 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83f9f740cdd9480da49b5a6a07552b4f)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:47.346593 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71912ec1-837b-4731-a675-e856266ef426 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:47.554608 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-71912ec1-837b-4731-a675-e856266ef426 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83f9f740cdd9480da49b5a6a07552b4f'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:47.565065 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-71912ec1-837b-4731-a675-e856266ef426 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83f9f740cdd9480da49b5a6a07552b4f'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:47.565826 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-71912ec1-837b-4731-a675-e856266ef426 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83f9f740cdd9480da49b5a6a07552b4f'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:47.566442 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-71912ec1-837b-4731-a675-e856266ef426 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83f9f740cdd9480da49b5a6a07552b4f'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:47.581446 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-71912ec1-837b-4731-a675-e856266ef426 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83f9f740cdd9480da49b5a6a07552b4f'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:47.653388 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1048/2063] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:27:47 2022] DELETE /identity/v3/users/83f9f740cdd9480da49b5a6a07552b4f => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:27:48.164313 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382280c6-e0ae-49eb-9c6e-4b561595e3fb admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:48.165903 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-382280c6-e0ae-49eb-9c6e-4b561595e3fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:48.180471 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382280c6-e0ae-49eb-9c6e-4b561595e3fb admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:48.243212 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f254c96-8bfe-40df-b5fd-9c03311ba08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:48.244469 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f254c96-8bfe-40df-b5fd-9c03311ba08e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:48.244744 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f254c96-8bfe-40df-b5fd-9c03311ba08e admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:48.245035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f254c96-8bfe-40df-b5fd-9c03311ba08e admin admin] PATH_INFO: `/v3/projects/9f03abcc62d64a489ca80bb90ca7fbdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:48.246250 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f254c96-8bfe-40df-b5fd-9c03311ba08e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f03abcc62d64a489ca80bb90ca7fbdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:48.248274 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f254c96-8bfe-40df-b5fd-9c03311ba08e admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:48.274643 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:48.366524 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6f254c96-8bfe-40df-b5fd-9c03311ba08e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f03abcc62d64a489ca80bb90ca7fbdc', 'request_id': 'req-6f254c96-8bfe-40df-b5fd-9c03311ba08e'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:48.372035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6f254c96-8bfe-40df-b5fd-9c03311ba08e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f03abcc62d64a489ca80bb90ca7fbdc', 'request_id': 'req-6f254c96-8bfe-40df-b5fd-9c03311ba08e'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:48.431436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6f254c96-8bfe-40df-b5fd-9c03311ba08e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f03abcc62d64a489ca80bb90ca7fbdc', 'request_id': 'req-6f254c96-8bfe-40df-b5fd-9c03311ba08e'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:48.444361 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6f254c96-8bfe-40df-b5fd-9c03311ba08e admin admin] The token cache is being invalidate because project 9f03abcc62d64a489ca80bb90ca7fbdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:27:48.444687 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6f254c96-8bfe-40df-b5fd-9c03311ba08e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:48.445361 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6f254c96-8bfe-40df-b5fd-9c03311ba08e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:27:48.447011 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1016/2064] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:27:48 2022] DELETE /identity/v3/projects/9f03abcc62d64a489ca80bb90ca7fbdc => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:27:48.452016 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc01f2-5c08-46be-a77d-36e8ac1244b4 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:48.452561 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc01f2-5c08-46be-a77d-36e8ac1244b4 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:48.452677 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc01f2-5c08-46be-a77d-36e8ac1244b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:48.471533 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-5fbc01f2-5c08-46be-a77d-36e8ac1244b4 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:48.472831 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-5fbc01f2-5c08-46be-a77d-36e8ac1244b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:48.538225 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1049/2065] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:27:48.546415 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b463f2-8f43-4ddf-ae8c-0debc6de4275 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:48.546718 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b463f2-8f43-4ddf-ae8c-0debc6de4275 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:48.547003 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b463f2-8f43-4ddf-ae8c-0debc6de4275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:48.565273 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-78b463f2-8f43-4ddf-ae8c-0debc6de4275 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:48.566538 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-78b463f2-8f43-4ddf-ae8c-0debc6de4275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:48.575425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1017/2066] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:27:48.580512 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc01f2-5c08-46be-a77d-36e8ac1244b4 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:48.590316 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc01f2-5c08-46be-a77d-36e8ac1244b4 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:48.595723 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd95c1-b37b-48ad-828d-4aa04f8c2ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:48.596380 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd95c1-b37b-48ad-828d-4aa04f8c2ff1 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:48.596700 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd95c1-b37b-48ad-828d-4aa04f8c2ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:48.596976 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd95c1-b37b-48ad-828d-4aa04f8c2ff1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:48.597712 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbd95c1-b37b-48ad-828d-4aa04f8c2ff1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:27:48.598008 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbd95c1-b37b-48ad-828d-4aa04f8c2ff1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:48.599912 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbd95c1-b37b-48ad-828d-4aa04f8c2ff1 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:48.612117 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1050/2067] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:27:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:27:48.617661 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b463f2-8f43-4ddf-ae8c-0debc6de4275 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:48.627727 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b463f2-8f43-4ddf-ae8c-0debc6de4275 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:48.632590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6578d38-aedd-41e8-a6c8-f25e179cdb30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:48.633254 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6578d38-aedd-41e8-a6c8-f25e179cdb30 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:48.633548 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6578d38-aedd-41e8-a6c8-f25e179cdb30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:48.633833 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6578d38-aedd-41e8-a6c8-f25e179cdb30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:48.634545 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6578d38-aedd-41e8-a6c8-f25e179cdb30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:48.636476 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6578d38-aedd-41e8-a6c8-f25e179cdb30 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:48.708673 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1018/2068] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:27:48 2022] POST /identity/v3/projects => generated 387 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:27:48.713209 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd95c1-b37b-48ad-828d-4aa04f8c2ff1 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:48.725014 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd95c1-b37b-48ad-828d-4aa04f8c2ff1 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:48.771104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:48.785091 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef335628-a18f-4535-a403-55e51d73f24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:48.785949 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef335628-a18f-4535-a403-55e51d73f24c admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:48.786235 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef335628-a18f-4535-a403-55e51d73f24c admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:48.786516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef335628-a18f-4535-a403-55e51d73f24c admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:48.787235 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef335628-a18f-4535-a403-55e51d73f24c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:48.789354 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef335628-a18f-4535-a403-55e51d73f24c admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:48.866914 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1051/2069] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:27:48 2022] POST /identity/v3/users => generated 294 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:27:48.878255 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6578d38-aedd-41e8-a6c8-f25e179cdb30 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:48.896964 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6578d38-aedd-41e8-a6c8-f25e179cdb30 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:48.902491 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2545b5-667e-4cd7-9eeb-3f98617ad9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:48.903179 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2545b5-667e-4cd7-9eeb-3f98617ad9db admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:48.903443 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2545b5-667e-4cd7-9eeb-3f98617ad9db admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:48.903824 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2545b5-667e-4cd7-9eeb-3f98617ad9db admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:48.904598 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2545b5-667e-4cd7-9eeb-3f98617ad9db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:48.906183 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2545b5-667e-4cd7-9eeb-3f98617ad9db admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:48.919073 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1019/2070] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:27:48.923007 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef335628-a18f-4535-a403-55e51d73f24c admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:48.933229 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef335628-a18f-4535-a403-55e51d73f24c admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:48.938860 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9376bb-13d4-48f8-a96f-1fb702cf9e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:48.939633 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9376bb-13d4-48f8-a96f-1fb702cf9e12 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:48.940546 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9376bb-13d4-48f8-a96f-1fb702cf9e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:48.940965 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9376bb-13d4-48f8-a96f-1fb702cf9e12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:48.941577 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9376bb-13d4-48f8-a96f-1fb702cf9e12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:48.943188 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9376bb-13d4-48f8-a96f-1fb702cf9e12 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:48.956490 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1052/2071] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:27:48.961074 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2545b5-667e-4cd7-9eeb-3f98617ad9db admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:48.971489 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2545b5-667e-4cd7-9eeb-3f98617ad9db admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:48.976921 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b971aa6-ac83-4008-90a3-1ca4fc997369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:48.977959 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b971aa6-ac83-4008-90a3-1ca4fc997369 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:48.978246 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b971aa6-ac83-4008-90a3-1ca4fc997369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:48.978547 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b971aa6-ac83-4008-90a3-1ca4fc997369 admin admin] PATH_INFO: `/v3/projects/8312bb22320f4e12a44acfc2d38b0464/users/29396460038a497abb6d7b63506aa465/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:48.997987 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b971aa6-ac83-4008-90a3-1ca4fc997369 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8312bb22320f4e12a44acfc2d38b0464, user_id=29396460038a497abb6d7b63506aa465, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:49.001388 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b971aa6-ac83-4008-90a3-1ca4fc997369 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:49.068282 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1020/2072] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:27:48 2022] PUT /identity/v3/projects/8312bb22320f4e12a44acfc2d38b0464/users/29396460038a497abb6d7b63506aa465/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:27:49.080586 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850335f-0c21-4b2d-ba42-4ef81438d166 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:49.080973 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850335f-0c21-4b2d-ba42-4ef81438d166 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:49.081250 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850335f-0c21-4b2d-ba42-4ef81438d166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:49.114562 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-a850335f-0c21-4b2d-ba42-4ef81438d166 None None] MFA Rules not processed for user `29396460038a497abb6d7b63506aa465`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:49.115878 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-a850335f-0c21-4b2d-ba42-4ef81438d166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:49.193520 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1053/2073] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:27:49.194861 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b971aa6-ac83-4008-90a3-1ca4fc997369 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:49.260939 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b971aa6-ac83-4008-90a3-1ca4fc997369 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:49.266395 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69d9a45-f586-4db8-b701-b0149c588b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:49.267202 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d9a45-f586-4db8-b701-b0149c588b7a admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:49.267494 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d9a45-f586-4db8-b701-b0149c588b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:49.267780 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d9a45-f586-4db8-b701-b0149c588b7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:49.268526 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69d9a45-f586-4db8-b701-b0149c588b7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:49.270472 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69d9a45-f586-4db8-b701-b0149c588b7a admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:49.358526 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1021/2074] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:27:49 2022] POST /identity/v3/projects => generated 387 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:27:49.362736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850335f-0c21-4b2d-ba42-4ef81438d166 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:49.376937 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850335f-0c21-4b2d-ba42-4ef81438d166 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:49.440616 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8cd869-7ae6-4bad-b1ac-aeca91bc5b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:49.441121 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8cd869-7ae6-4bad-b1ac-aeca91bc5b31 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:49.441406 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8cd869-7ae6-4bad-b1ac-aeca91bc5b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:49.441686 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8cd869-7ae6-4bad-b1ac-aeca91bc5b31 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:49.442413 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8cd869-7ae6-4bad-b1ac-aeca91bc5b31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:49.444354 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8cd869-7ae6-4bad-b1ac-aeca91bc5b31 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:49.540595 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1054/2075] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:27:49 2022] POST /identity/v3/users => generated 294 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:27:49.544902 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69d9a45-f586-4db8-b701-b0149c588b7a admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:49.555947 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69d9a45-f586-4db8-b701-b0149c588b7a admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:49.560736 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029789bb-f7fa-45f6-b988-22c7352e4f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:49.562547 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029789bb-f7fa-45f6-b988-22c7352e4f15 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:49.562547 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029789bb-f7fa-45f6-b988-22c7352e4f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:49.562547 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029789bb-f7fa-45f6-b988-22c7352e4f15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:49.562999 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029789bb-f7fa-45f6-b988-22c7352e4f15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:49.564378 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029789bb-f7fa-45f6-b988-22c7352e4f15 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:49.576706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1022/2076] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:27:49.589103 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8cd869-7ae6-4bad-b1ac-aeca91bc5b31 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:49.600854 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8cd869-7ae6-4bad-b1ac-aeca91bc5b31 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:49.605599 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5856dcb-5944-461b-a4a5-04710ff49acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:49.606328 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5856dcb-5944-461b-a4a5-04710ff49acf admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:49.606617 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5856dcb-5944-461b-a4a5-04710ff49acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:49.606895 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5856dcb-5944-461b-a4a5-04710ff49acf admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:49.607629 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5856dcb-5944-461b-a4a5-04710ff49acf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:49.609284 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5856dcb-5944-461b-a4a5-04710ff49acf admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:49.620506 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1055/2077] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:27:49.625005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029789bb-f7fa-45f6-b988-22c7352e4f15 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:49.635688 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029789bb-f7fa-45f6-b988-22c7352e4f15 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:49.640519 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca0c0a-c389-47f6-b778-6a55321b0df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:49.642143 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca0c0a-c389-47f6-b778-6a55321b0df2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:49.642438 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca0c0a-c389-47f6-b778-6a55321b0df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:49.643162 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca0c0a-c389-47f6-b778-6a55321b0df2 admin admin] PATH_INFO: `/v3/projects/8d4b8954dd3a4752bdebbad956036925/users/481b2bb87e134fe09ca156a9b30c99b1/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:49.660536 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ca0c0a-c389-47f6-b778-6a55321b0df2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d4b8954dd3a4752bdebbad956036925, user_id=481b2bb87e134fe09ca156a9b30c99b1, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:49.663359 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ca0c0a-c389-47f6-b778-6a55321b0df2 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:49.665712 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:49.734128 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1023/2078] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:27:49 2022] PUT /identity/v3/projects/8d4b8954dd3a4752bdebbad956036925/users/481b2bb87e134fe09ca156a9b30c99b1/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:27:49.747070 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed9be3-5a1e-4a0c-987b-fce3559b9343 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:49.747365 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed9be3-5a1e-4a0c-987b-fce3559b9343 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:49.747689 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed9be3-5a1e-4a0c-987b-fce3559b9343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:49.776813 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-11ed9be3-5a1e-4a0c-987b-fce3559b9343 None None] MFA Rules not processed for user `481b2bb87e134fe09ca156a9b30c99b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:49.778132 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-11ed9be3-5a1e-4a0c-987b-fce3559b9343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:49.800194 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:49.846421 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1056/2079] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:27:49.851063 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca0c0a-c389-47f6-b778-6a55321b0df2 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:49.923008 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca0c0a-c389-47f6-b778-6a55321b0df2 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:49.927893 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c4d510-8f03-4c77-a021-7878c8a7bc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:49.928821 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4d510-8f03-4c77-a021-7878c8a7bc46 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:49.929108 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4d510-8f03-4c77-a021-7878c8a7bc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:49.929397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4d510-8f03-4c77-a021-7878c8a7bc46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:49.930096 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c4d510-8f03-4c77-a021-7878c8a7bc46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:49.932018 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c4d510-8f03-4c77-a021-7878c8a7bc46 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:50.050163 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1024/2080] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:27:49 2022] POST /identity/v3/projects => generated 387 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:27:50.054208 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed9be3-5a1e-4a0c-987b-fce3559b9343 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:50.065417 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed9be3-5a1e-4a0c-987b-fce3559b9343 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:50.126944 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc2aef8-c40a-4ed8-9c34-4997066d8b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:50.127625 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2aef8-c40a-4ed8-9c34-4997066d8b31 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:50.127919 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2aef8-c40a-4ed8-9c34-4997066d8b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:50.128201 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2aef8-c40a-4ed8-9c34-4997066d8b31 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:50.129004 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc2aef8-c40a-4ed8-9c34-4997066d8b31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:50.130933 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc2aef8-c40a-4ed8-9c34-4997066d8b31 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:50.232538 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1057/2081] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:27:50 2022] POST /identity/v3/users => generated 300 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:27:50.236510 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c4d510-8f03-4c77-a021-7878c8a7bc46 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:50.247726 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c4d510-8f03-4c77-a021-7878c8a7bc46 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:50.253589 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fbc142-8b48-402d-8c0d-01822b0d809c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:50.254279 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fbc142-8b48-402d-8c0d-01822b0d809c admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:50.254572 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fbc142-8b48-402d-8c0d-01822b0d809c admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:50.255061 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fbc142-8b48-402d-8c0d-01822b0d809c admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:50.255925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fbc142-8b48-402d-8c0d-01822b0d809c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:50.257866 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fbc142-8b48-402d-8c0d-01822b0d809c admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:50.277314 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1025/2082] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:27:50.283599 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc2aef8-c40a-4ed8-9c34-4997066d8b31 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:50.294025 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc2aef8-c40a-4ed8-9c34-4997066d8b31 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:50.299138 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c9d17-aa76-4a0e-93b0-1b02859500c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:50.300003 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c9d17-aa76-4a0e-93b0-1b02859500c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:50.300288 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c9d17-aa76-4a0e-93b0-1b02859500c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:50.300601 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c9d17-aa76-4a0e-93b0-1b02859500c6 admin admin] PATH_INFO: `/v3/projects/d155e072db9a455da60f82555ace0759/users/e47ec46bbfcd4568be4003777bd9efca/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:50.317046 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3c9d17-aa76-4a0e-93b0-1b02859500c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d155e072db9a455da60f82555ace0759, user_id=e47ec46bbfcd4568be4003777bd9efca, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:50.319844 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3c9d17-aa76-4a0e-93b0-1b02859500c6 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:50.403511 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1058/2083] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:27:50 2022] PUT /identity/v3/projects/d155e072db9a455da60f82555ace0759/users/e47ec46bbfcd4568be4003777bd9efca/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:27:50.408452 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fbc142-8b48-402d-8c0d-01822b0d809c admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:50.419401 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fbc142-8b48-402d-8c0d-01822b0d809c admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:50.480256 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84db1a7-c1e7-4ed3-b819-b668965332c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:50.480877 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84db1a7-c1e7-4ed3-b819-b668965332c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:50.481207 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84db1a7-c1e7-4ed3-b819-b668965332c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:50.481492 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84db1a7-c1e7-4ed3-b819-b668965332c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:50.482225 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84db1a7-c1e7-4ed3-b819-b668965332c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:50.483826 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84db1a7-c1e7-4ed3-b819-b668965332c3 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:50.495729 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1026/2084] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 06:27:50.500557 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c9d17-aa76-4a0e-93b0-1b02859500c6 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:50.512717 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c9d17-aa76-4a0e-93b0-1b02859500c6 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:50.518339 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94e116-3213-4214-8dac-6d1350403be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:50.519284 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94e116-3213-4214-8dac-6d1350403be4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:50.519581 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94e116-3213-4214-8dac-6d1350403be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:50.520193 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94e116-3213-4214-8dac-6d1350403be4 admin admin] PATH_INFO: `/v3/projects/d155e072db9a455da60f82555ace0759/users/e47ec46bbfcd4568be4003777bd9efca/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:50.522112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e94e116-3213-4214-8dac-6d1350403be4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d155e072db9a455da60f82555ace0759, user_id=e47ec46bbfcd4568be4003777bd9efca, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:50.524877 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e94e116-3213-4214-8dac-6d1350403be4 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:50.540109 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1059/2085] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:27:50 2022] PUT /identity/v3/projects/d155e072db9a455da60f82555ace0759/users/e47ec46bbfcd4568be4003777bd9efca/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:27:50.552648 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfeb095-ee92-467e-9323-55b5efb226ef None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:50.553018 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfeb095-ee92-467e-9323-55b5efb226ef None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:50.553263 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfeb095-ee92-467e-9323-55b5efb226ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:50.586149 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-fcfeb095-ee92-467e-9323-55b5efb226ef None None] MFA Rules not processed for user `e47ec46bbfcd4568be4003777bd9efca`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:27:50.589015 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-fcfeb095-ee92-467e-9323-55b5efb226ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:50.644904 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:27:50.674085 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1027/2086] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:27:50 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:27:50.680187 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94e116-3213-4214-8dac-6d1350403be4 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:50.684858 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4e94e116-3213-4214-8dac-6d1350403be4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:27:50.750825 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94e116-3213-4214-8dac-6d1350403be4 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:50.754951 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca187751-509a-4ae8-9475-fd1a3423384a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:50.755643 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca187751-509a-4ae8-9475-fd1a3423384a service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:50.755932 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca187751-509a-4ae8-9475-fd1a3423384a service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:50.756207 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca187751-509a-4ae8-9475-fd1a3423384a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:50.767282 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca187751-509a-4ae8-9475-fd1a3423384a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:50.768910 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca187751-509a-4ae8-9475-fd1a3423384a service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:50.779737 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1060/2087] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:27:50 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 06:27:51.815798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfeb095-ee92-467e-9323-55b5efb226ef None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:27:51.825345 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfeb095-ee92-467e-9323-55b5efb226ef None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:27:51.829054 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0baaf4-ab97-4957-b4e9-2f01015c7096 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:27:51.829697 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0baaf4-ab97-4957-b4e9-2f01015c7096 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:27:51.829984 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0baaf4-ab97-4957-b4e9-2f01015c7096 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:27:51.830268 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0baaf4-ab97-4957-b4e9-2f01015c7096 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:27:51.909212 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0baaf4-ab97-4957-b4e9-2f01015c7096 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:27:51.910660 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0baaf4-ab97-4957-b4e9-2f01015c7096 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:27:51.919975 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1028/2088] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:27:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:28:00.774705 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=87,keystone:INSERT=5 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:28:01.915784 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=33 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:28:09.744265 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca187751-509a-4ae8-9475-fd1a3423384a service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:09.754722 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca187751-509a-4ae8-9475-fd1a3423384a service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:09.759225 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3d6f49-eef6-40ab-8ffc-7b84c92b1e0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:09.759924 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d6f49-eef6-40ab-8ffc-7b84c92b1e0c service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:09.760207 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d6f49-eef6-40ab-8ffc-7b84c92b1e0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:09.760523 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d6f49-eef6-40ab-8ffc-7b84c92b1e0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:09.874636 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3d6f49-eef6-40ab-8ffc-7b84c92b1e0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:09.878818 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3d6f49-eef6-40ab-8ffc-7b84c92b1e0c service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:09.899003 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1061/2089] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:28:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:28:16.368971 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0baaf4-ab97-4957-b4e9-2f01015c7096 service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:16.473509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0baaf4-ab97-4957-b4e9-2f01015c7096 service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:16.478303 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3fddca-8f05-4582-ae8e-6427b5189bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:16.479217 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3fddca-8f05-4582-ae8e-6427b5189bba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:16.479506 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3fddca-8f05-4582-ae8e-6427b5189bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:16.479780 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3fddca-8f05-4582-ae8e-6427b5189bba admin admin] PATH_INFO: `/v3/users/29396460038a497abb6d7b63506aa465` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:16.481162 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3fddca-8f05-4582-ae8e-6427b5189bba admin admin] RBAC: Authorizing `identity:delete_user(user_id=29396460038a497abb6d7b63506aa465)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:16.483128 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3fddca-8f05-4582-ae8e-6427b5189bba admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:16.680387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3d3fddca-8f05-4582-ae8e-6427b5189bba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29396460038a497abb6d7b63506aa465'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:16.693281 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3d3fddca-8f05-4582-ae8e-6427b5189bba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29396460038a497abb6d7b63506aa465'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:16.710161 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3d3fddca-8f05-4582-ae8e-6427b5189bba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29396460038a497abb6d7b63506aa465'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:16.710949 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3d3fddca-8f05-4582-ae8e-6427b5189bba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29396460038a497abb6d7b63506aa465'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:16.711575 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3d3fddca-8f05-4582-ae8e-6427b5189bba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29396460038a497abb6d7b63506aa465'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:16.806033 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1029/2090] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:28:16 2022] DELETE /identity/v3/users/29396460038a497abb6d7b63506aa465 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:28:16.812531 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef37eb7-8f30-4c35-a3e1-832e7be9dc71 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:16.813323 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef37eb7-8f30-4c35-a3e1-832e7be9dc71 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:16.813542 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef37eb7-8f30-4c35-a3e1-832e7be9dc71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:16.840987 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-7ef37eb7-8f30-4c35-a3e1-832e7be9dc71 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:28:16.843702 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-7ef37eb7-8f30-4c35-a3e1-832e7be9dc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:16.975482 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1062/2091] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:28:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:28:16.994623 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3fddca-8f05-4582-ae8e-6427b5189bba admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:16.994623 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-3d3fddca-8f05-4582-ae8e-6427b5189bba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:17.095164 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3fddca-8f05-4582-ae8e-6427b5189bba admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:17.100189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e58b5-cdfb-44d4-8cd7-099ff93c358c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:17.101243 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e58b5-cdfb-44d4-8cd7-099ff93c358c service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:17.101842 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e58b5-cdfb-44d4-8cd7-099ff93c358c service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:17.102882 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e58b5-cdfb-44d4-8cd7-099ff93c358c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:17.122735 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663e58b5-cdfb-44d4-8cd7-099ff93c358c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:17.124670 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663e58b5-cdfb-44d4-8cd7-099ff93c358c service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:17.140168 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1030/2092] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:28:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:28:17.776212 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef37eb7-8f30-4c35-a3e1-832e7be9dc71 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:17.778462 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-7ef37eb7-8f30-4c35-a3e1-832e7be9dc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:17.793855 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef37eb7-8f30-4c35-a3e1-832e7be9dc71 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:17.800655 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff74d22-7d21-411b-a5e0-3fa1bdcec29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:17.801551 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff74d22-7d21-411b-a5e0-3fa1bdcec29a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:17.801893 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff74d22-7d21-411b-a5e0-3fa1bdcec29a admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:17.802098 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff74d22-7d21-411b-a5e0-3fa1bdcec29a admin admin] PATH_INFO: `/v3/projects/8312bb22320f4e12a44acfc2d38b0464` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:17.804066 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff74d22-7d21-411b-a5e0-3fa1bdcec29a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8312bb22320f4e12a44acfc2d38b0464)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:17.806065 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff74d22-7d21-411b-a5e0-3fa1bdcec29a admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:18.066336 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-3ff74d22-7d21-411b-a5e0-3fa1bdcec29a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8312bb22320f4e12a44acfc2d38b0464', 'request_id': 'req-3ff74d22-7d21-411b-a5e0-3fa1bdcec29a'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:18.075229 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-3ff74d22-7d21-411b-a5e0-3fa1bdcec29a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8312bb22320f4e12a44acfc2d38b0464', 'request_id': 'req-3ff74d22-7d21-411b-a5e0-3fa1bdcec29a'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:18.212856 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-3ff74d22-7d21-411b-a5e0-3fa1bdcec29a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8312bb22320f4e12a44acfc2d38b0464', 'request_id': 'req-3ff74d22-7d21-411b-a5e0-3fa1bdcec29a'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:18.232306 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-3ff74d22-7d21-411b-a5e0-3fa1bdcec29a admin admin] The token cache is being invalidate because project 8312bb22320f4e12a44acfc2d38b0464 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:28:18.233244 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-3ff74d22-7d21-411b-a5e0-3fa1bdcec29a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:18.235109 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-3ff74d22-7d21-411b-a5e0-3fa1bdcec29a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:18.237680 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1063/2093] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:28:17 2022] DELETE /identity/v3/projects/8312bb22320f4e12a44acfc2d38b0464 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:28:18.242801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e58b5-cdfb-44d4-8cd7-099ff93c358c service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:18.255509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e58b5-cdfb-44d4-8cd7-099ff93c358c service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:18.297433 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:28:18.393164 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a7890b-2113-4636-a113-4b59a11b4a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:18.394748 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7890b-2113-4636-a113-4b59a11b4a2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:18.395421 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7890b-2113-4636-a113-4b59a11b4a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:18.396001 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7890b-2113-4636-a113-4b59a11b4a2f admin admin] PATH_INFO: `/v3/users/481b2bb87e134fe09ca156a9b30c99b1` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:18.398460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a7890b-2113-4636-a113-4b59a11b4a2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=481b2bb87e134fe09ca156a9b30c99b1)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:18.402674 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a7890b-2113-4636-a113-4b59a11b4a2f admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:18.700778 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-c4a7890b-2113-4636-a113-4b59a11b4a2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '481b2bb87e134fe09ca156a9b30c99b1'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:18.710856 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-c4a7890b-2113-4636-a113-4b59a11b4a2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '481b2bb87e134fe09ca156a9b30c99b1'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:18.730830 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-c4a7890b-2113-4636-a113-4b59a11b4a2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '481b2bb87e134fe09ca156a9b30c99b1'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:18.731960 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-c4a7890b-2113-4636-a113-4b59a11b4a2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '481b2bb87e134fe09ca156a9b30c99b1'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:18.733231 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-c4a7890b-2113-4636-a113-4b59a11b4a2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '481b2bb87e134fe09ca156a9b30c99b1'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:18.854643 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1031/2094] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:28:18 2022] DELETE /identity/v3/users/481b2bb87e134fe09ca156a9b30c99b1 => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:28:19.742400 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff74d22-7d21-411b-a5e0-3fa1bdcec29a admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:19.745521 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-3ff74d22-7d21-411b-a5e0-3fa1bdcec29a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:19.759864 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff74d22-7d21-411b-a5e0-3fa1bdcec29a admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:19.785139 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:28:19.837414 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae475f98-36d8-4c83-88cd-d789354a041c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:19.838391 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae475f98-36d8-4c83-88cd-d789354a041c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:19.838738 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae475f98-36d8-4c83-88cd-d789354a041c admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:19.839081 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae475f98-36d8-4c83-88cd-d789354a041c admin admin] PATH_INFO: `/v3/projects/8d4b8954dd3a4752bdebbad956036925` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:19.840584 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae475f98-36d8-4c83-88cd-d789354a041c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d4b8954dd3a4752bdebbad956036925)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:19.842887 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae475f98-36d8-4c83-88cd-d789354a041c admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:20.023075 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ae475f98-36d8-4c83-88cd-d789354a041c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d4b8954dd3a4752bdebbad956036925', 'request_id': 'req-ae475f98-36d8-4c83-88cd-d789354a041c'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:20.033685 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ae475f98-36d8-4c83-88cd-d789354a041c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d4b8954dd3a4752bdebbad956036925', 'request_id': 'req-ae475f98-36d8-4c83-88cd-d789354a041c'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:20.151824 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ae475f98-36d8-4c83-88cd-d789354a041c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d4b8954dd3a4752bdebbad956036925', 'request_id': 'req-ae475f98-36d8-4c83-88cd-d789354a041c'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:20.166698 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ae475f98-36d8-4c83-88cd-d789354a041c admin admin] The token cache is being invalidate because project 8d4b8954dd3a4752bdebbad956036925 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:28:20.166993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ae475f98-36d8-4c83-88cd-d789354a041c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:20.167676 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ae475f98-36d8-4c83-88cd-d789354a041c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:20.169383 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1064/2095] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:28:19 2022] DELETE /identity/v3/projects/8d4b8954dd3a4752bdebbad956036925 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:28:20.174131 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a7890b-2113-4636-a113-4b59a11b4a2f admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:20.176024 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-c4a7890b-2113-4636-a113-4b59a11b4a2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:20.191859 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a7890b-2113-4636-a113-4b59a11b4a2f admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:20.272429 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe6632-76e7-4bba-b2d1-eb4988711a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:20.272429 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe6632-76e7-4bba-b2d1-eb4988711a43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:20.272429 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe6632-76e7-4bba-b2d1-eb4988711a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:20.272429 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe6632-76e7-4bba-b2d1-eb4988711a43 admin admin] PATH_INFO: `/v3/users/e47ec46bbfcd4568be4003777bd9efca` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:20.273348 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fe6632-76e7-4bba-b2d1-eb4988711a43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e47ec46bbfcd4568be4003777bd9efca)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:20.275454 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fe6632-76e7-4bba-b2d1-eb4988711a43 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:20.514954 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:28:20.519694 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-47fe6632-76e7-4bba-b2d1-eb4988711a43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e47ec46bbfcd4568be4003777bd9efca'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:20.536870 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-47fe6632-76e7-4bba-b2d1-eb4988711a43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e47ec46bbfcd4568be4003777bd9efca'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:20.569120 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-47fe6632-76e7-4bba-b2d1-eb4988711a43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e47ec46bbfcd4568be4003777bd9efca'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:20.570666 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-47fe6632-76e7-4bba-b2d1-eb4988711a43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e47ec46bbfcd4568be4003777bd9efca'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:20.571488 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-47fe6632-76e7-4bba-b2d1-eb4988711a43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e47ec46bbfcd4568be4003777bd9efca'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:20.677407 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1032/2096] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:28:20 2022] DELETE /identity/v3/users/e47ec46bbfcd4568be4003777bd9efca => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:28:21.362459 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae475f98-36d8-4c83-88cd-d789354a041c admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:21.364982 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ae475f98-36d8-4c83-88cd-d789354a041c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:21.390213 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae475f98-36d8-4c83-88cd-d789354a041c admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:21.496000 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223cb6c8-f4a3-429a-a30c-32c1ada1209d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:21.496950 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223cb6c8-f4a3-429a-a30c-32c1ada1209d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:21.497229 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223cb6c8-f4a3-429a-a30c-32c1ada1209d admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:21.497514 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223cb6c8-f4a3-429a-a30c-32c1ada1209d admin admin] PATH_INFO: `/v3/projects/d155e072db9a455da60f82555ace0759` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:21.498802 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223cb6c8-f4a3-429a-a30c-32c1ada1209d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d155e072db9a455da60f82555ace0759)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:21.501068 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223cb6c8-f4a3-429a-a30c-32c1ada1209d admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:21.680692 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-223cb6c8-f4a3-429a-a30c-32c1ada1209d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd155e072db9a455da60f82555ace0759', 'request_id': 'req-223cb6c8-f4a3-429a-a30c-32c1ada1209d'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:21.693172 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-223cb6c8-f4a3-429a-a30c-32c1ada1209d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd155e072db9a455da60f82555ace0759', 'request_id': 'req-223cb6c8-f4a3-429a-a30c-32c1ada1209d'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:21.696354 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:28:21.854195 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-223cb6c8-f4a3-429a-a30c-32c1ada1209d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd155e072db9a455da60f82555ace0759', 'request_id': 'req-223cb6c8-f4a3-429a-a30c-32c1ada1209d'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:21.865459 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-223cb6c8-f4a3-429a-a30c-32c1ada1209d admin admin] The token cache is being invalidate because project d155e072db9a455da60f82555ace0759 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:28:21.865949 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-223cb6c8-f4a3-429a-a30c-32c1ada1209d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:21.866962 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-223cb6c8-f4a3-429a-a30c-32c1ada1209d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:21.868936 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1065/2097] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:28:21 2022] DELETE /identity/v3/projects/d155e072db9a455da60f82555ace0759 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:28:21.875971 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd581c-601a-4c67-b1a8-bb8541d1d064 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:21.876479 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd581c-601a-4c67-b1a8-bb8541d1d064 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:21.876569 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd581c-601a-4c67-b1a8-bb8541d1d064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:21.896351 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-1dfd581c-601a-4c67-b1a8-bb8541d1d064 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:28:21.897519 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-1dfd581c-601a-4c67-b1a8-bb8541d1d064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:21.960473 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1033/2098] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:28:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:28:21.971677 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf320eae-6331-4d30-a947-c2a6547745ff None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:21.972104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf320eae-6331-4d30-a947-c2a6547745ff None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:21.972245 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf320eae-6331-4d30-a947-c2a6547745ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:21.991723 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-bf320eae-6331-4d30-a947-c2a6547745ff None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:28:21.992976 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-bf320eae-6331-4d30-a947-c2a6547745ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:22.003871 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1066/2099] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:28:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:28:22.009313 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd581c-601a-4c67-b1a8-bb8541d1d064 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:22.023796 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd581c-601a-4c67-b1a8-bb8541d1d064 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:22.028481 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce554c2-926a-45f2-9d79-be0452392694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:22.029143 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce554c2-926a-45f2-9d79-be0452392694 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:22.029433 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce554c2-926a-45f2-9d79-be0452392694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:22.029712 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce554c2-926a-45f2-9d79-be0452392694 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:22.030577 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce554c2-926a-45f2-9d79-be0452392694 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:28:22.030895 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce554c2-926a-45f2-9d79-be0452392694 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:22.033061 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce554c2-926a-45f2-9d79-be0452392694 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:22.049695 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1034/2100] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:28:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:28:22.051181 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf320eae-6331-4d30-a947-c2a6547745ff None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:22.060972 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf320eae-6331-4d30-a947-c2a6547745ff None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:22.066653 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261528ff-e4e9-4b30-9806-f15a7da6cf4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:22.067351 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261528ff-e4e9-4b30-9806-f15a7da6cf4c admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:22.067655 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261528ff-e4e9-4b30-9806-f15a7da6cf4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:22.067963 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261528ff-e4e9-4b30-9806-f15a7da6cf4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:22.068820 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261528ff-e4e9-4b30-9806-f15a7da6cf4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:22.070953 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261528ff-e4e9-4b30-9806-f15a7da6cf4c admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:22.211292 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1067/2101] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:28:22 2022] POST /identity/v3/projects => generated 391 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:28:22.215747 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce554c2-926a-45f2-9d79-be0452392694 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:22.226909 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce554c2-926a-45f2-9d79-be0452392694 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:22.292716 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade5425f-dd4e-4226-8775-b6af6f609757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:22.293401 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5425f-dd4e-4226-8775-b6af6f609757 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:22.293686 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5425f-dd4e-4226-8775-b6af6f609757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:22.293962 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5425f-dd4e-4226-8775-b6af6f609757 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:22.294669 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade5425f-dd4e-4226-8775-b6af6f609757 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:22.296657 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade5425f-dd4e-4226-8775-b6af6f609757 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:22.395117 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1035/2102] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:28:22 2022] POST /identity/v3/users => generated 296 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:28:22.408282 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261528ff-e4e9-4b30-9806-f15a7da6cf4c admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:22.438921 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261528ff-e4e9-4b30-9806-f15a7da6cf4c admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:22.448151 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c33177-f3dc-40b6-a45f-058f2e9afd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:22.449572 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c33177-f3dc-40b6-a45f-058f2e9afd47 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:22.450169 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c33177-f3dc-40b6-a45f-058f2e9afd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:22.451018 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c33177-f3dc-40b6-a45f-058f2e9afd47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:22.452515 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c33177-f3dc-40b6-a45f-058f2e9afd47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:22.455982 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c33177-f3dc-40b6-a45f-058f2e9afd47 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:22.481973 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1068/2103] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:28:22.483676 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade5425f-dd4e-4226-8775-b6af6f609757 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:22.518717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade5425f-dd4e-4226-8775-b6af6f609757 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:22.529608 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4cf73e-9c63-4ee8-809d-fbadedceb161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:22.530864 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4cf73e-9c63-4ee8-809d-fbadedceb161 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:22.531550 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4cf73e-9c63-4ee8-809d-fbadedceb161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:22.532172 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4cf73e-9c63-4ee8-809d-fbadedceb161 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:22.533874 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4cf73e-9c63-4ee8-809d-fbadedceb161 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:22.536873 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4cf73e-9c63-4ee8-809d-fbadedceb161 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:22.551639 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1036/2104] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:28:22.557876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c33177-f3dc-40b6-a45f-058f2e9afd47 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:22.576008 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c33177-f3dc-40b6-a45f-058f2e9afd47 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:22.584446 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25318886-0d4a-4417-9c8f-3f402208a136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:22.586004 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25318886-0d4a-4417-9c8f-3f402208a136 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:22.586458 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25318886-0d4a-4417-9c8f-3f402208a136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:22.586889 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25318886-0d4a-4417-9c8f-3f402208a136 admin admin] PATH_INFO: `/v3/projects/ecb6ea27e82944059022e671cd651978/users/d2c771a986f4453a9f32242983c478d4/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:22.614240 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25318886-0d4a-4417-9c8f-3f402208a136 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecb6ea27e82944059022e671cd651978, user_id=d2c771a986f4453a9f32242983c478d4, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:22.619303 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25318886-0d4a-4417-9c8f-3f402208a136 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:22.707133 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1069/2105] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:28:22 2022] PUT /identity/v3/projects/ecb6ea27e82944059022e671cd651978/users/d2c771a986f4453a9f32242983c478d4/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:28:22.719548 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d74de-f8ef-4d09-ac3f-8d845a8c882e None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:22.719829 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d74de-f8ef-4d09-ac3f-8d845a8c882e None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:22.720104 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d74de-f8ef-4d09-ac3f-8d845a8c882e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:22.762696 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-b10d74de-f8ef-4d09-ac3f-8d845a8c882e None None] MFA Rules not processed for user `d2c771a986f4453a9f32242983c478d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:28:22.764871 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b10d74de-f8ef-4d09-ac3f-8d845a8c882e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:22.788683 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:28:22.875382 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1037/2106] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:28:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:28:22.881828 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25318886-0d4a-4417-9c8f-3f402208a136 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:22.966322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25318886-0d4a-4417-9c8f-3f402208a136 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:22.974354 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad843ed-f256-4a54-8eed-12d061d40289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:22.975216 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad843ed-f256-4a54-8eed-12d061d40289 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:22.975590 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad843ed-f256-4a54-8eed-12d061d40289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:22.976006 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad843ed-f256-4a54-8eed-12d061d40289 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:22.976875 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad843ed-f256-4a54-8eed-12d061d40289 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:22.980339 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad843ed-f256-4a54-8eed-12d061d40289 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:23.079741 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1070/2107] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:28:22 2022] POST /identity/v3/projects => generated 393 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:28:23.084073 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10d74de-f8ef-4d09-ac3f-8d845a8c882e None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:23.098263 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10d74de-f8ef-4d09-ac3f-8d845a8c882e None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:23.174490 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f64ff1-94c1-4f34-850d-691729d38946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:23.175304 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f64ff1-94c1-4f34-850d-691729d38946 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:23.175586 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f64ff1-94c1-4f34-850d-691729d38946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:23.175860 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f64ff1-94c1-4f34-850d-691729d38946 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:23.176591 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f64ff1-94c1-4f34-850d-691729d38946 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:23.178806 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f64ff1-94c1-4f34-850d-691729d38946 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:23.348282 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1038/2108] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:28:23 2022] POST /identity/v3/users => generated 303 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:28:23.355738 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad843ed-f256-4a54-8eed-12d061d40289 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:23.370723 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad843ed-f256-4a54-8eed-12d061d40289 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:23.375787 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2e53c-a30c-45e1-9942-72ae0a07957d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:23.376615 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2e53c-a30c-45e1-9942-72ae0a07957d admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:23.376906 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2e53c-a30c-45e1-9942-72ae0a07957d admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:23.377188 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2e53c-a30c-45e1-9942-72ae0a07957d admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:23.378011 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad2e53c-a30c-45e1-9942-72ae0a07957d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:23.379728 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad2e53c-a30c-45e1-9942-72ae0a07957d admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:23.392632 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1071/2109] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:28:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:28:23.397175 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f64ff1-94c1-4f34-850d-691729d38946 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:23.409064 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f64ff1-94c1-4f34-850d-691729d38946 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:23.414404 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d291412-6e8c-4c07-a11f-603fe655b275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:23.415880 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d291412-6e8c-4c07-a11f-603fe655b275 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:23.416210 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d291412-6e8c-4c07-a11f-603fe655b275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:23.416577 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d291412-6e8c-4c07-a11f-603fe655b275 admin admin] PATH_INFO: `/v3/projects/e013d04313824ca798b86a0352d8afdd/users/ab0730b770254e7b8d4af4c3fd26003e/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:23.433133 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d291412-6e8c-4c07-a11f-603fe655b275 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e013d04313824ca798b86a0352d8afdd, user_id=ab0730b770254e7b8d4af4c3fd26003e, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:23.435936 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d291412-6e8c-4c07-a11f-603fe655b275 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:23.525144 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1039/2110] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:28:23 2022] PUT /identity/v3/projects/e013d04313824ca798b86a0352d8afdd/users/ab0730b770254e7b8d4af4c3fd26003e/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:28:23.529626 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2e53c-a30c-45e1-9942-72ae0a07957d admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:23.540499 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2e53c-a30c-45e1-9942-72ae0a07957d admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:23.634707 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:28:23.663587 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d22cfd1-fee6-4131-809e-f7f496bc3de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:23.664496 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22cfd1-fee6-4131-809e-f7f496bc3de8 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:23.665687 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22cfd1-fee6-4131-809e-f7f496bc3de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:23.665687 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22cfd1-fee6-4131-809e-f7f496bc3de8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:23.666019 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d22cfd1-fee6-4131-809e-f7f496bc3de8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:23.669241 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d22cfd1-fee6-4131-809e-f7f496bc3de8 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:23.681244 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1072/2111] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:28:23 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 06:28:23.685681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d291412-6e8c-4c07-a11f-603fe655b275 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:23.705084 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d291412-6e8c-4c07-a11f-603fe655b275 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:23.712266 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a78a94-39e3-4310-98b5-bf1a82fddd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:23.713217 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a78a94-39e3-4310-98b5-bf1a82fddd20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:23.713542 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a78a94-39e3-4310-98b5-bf1a82fddd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:23.713829 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a78a94-39e3-4310-98b5-bf1a82fddd20 admin admin] PATH_INFO: `/v3/projects/e013d04313824ca798b86a0352d8afdd/users/ab0730b770254e7b8d4af4c3fd26003e/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:23.715991 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a78a94-39e3-4310-98b5-bf1a82fddd20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e013d04313824ca798b86a0352d8afdd, user_id=ab0730b770254e7b8d4af4c3fd26003e, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:23.719430 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a78a94-39e3-4310-98b5-bf1a82fddd20 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:23.731041 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1040/2112] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:28:23 2022] PUT /identity/v3/projects/e013d04313824ca798b86a0352d8afdd/users/ab0730b770254e7b8d4af4c3fd26003e/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:28:23.742893 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9c40a-1cb1-4e31-9534-ecaf815eaca8 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:23.743233 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9c40a-1cb1-4e31-9534-ecaf815eaca8 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:23.743474 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9c40a-1cb1-4e31-9534-ecaf815eaca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:23.783051 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-01a9c40a-1cb1-4e31-9534-ecaf815eaca8 None None] MFA Rules not processed for user `ab0730b770254e7b8d4af4c3fd26003e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:28:23.784554 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-01a9c40a-1cb1-4e31-9534-ecaf815eaca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:23.863432 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1073/2113] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:28:23 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:28:23.875292 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a78a94-39e3-4310-98b5-bf1a82fddd20 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:23.877383 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-88a78a94-39e3-4310-98b5-bf1a82fddd20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:23.943876 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a78a94-39e3-4310-98b5-bf1a82fddd20 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:23.947649 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced39094-e427-4b21-b446-99fae0a52853 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:23.948386 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced39094-e427-4b21-b446-99fae0a52853 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:23.948702 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced39094-e427-4b21-b446-99fae0a52853 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:23.948997 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced39094-e427-4b21-b446-99fae0a52853 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:23.960400 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced39094-e427-4b21-b446-99fae0a52853 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:23.963732 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced39094-e427-4b21-b446-99fae0a52853 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:23.976444 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1041/2114] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:28:23 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:28:24.035402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a9c40a-1cb1-4e31-9534-ecaf815eaca8 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:24.036778 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-01a9c40a-1cb1-4e31-9534-ecaf815eaca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:24.049009 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a9c40a-1cb1-4e31-9534-ecaf815eaca8 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:24.052922 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c45e9b-3c50-433c-bd9c-be054324e1b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:24.053568 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c45e9b-3c50-433c-bd9c-be054324e1b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:24.053858 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c45e9b-3c50-433c-bd9c-be054324e1b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:24.054142 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c45e9b-3c50-433c-bd9c-be054324e1b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:24.113471 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c45e9b-3c50-433c-bd9c-be054324e1b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:24.114976 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c45e9b-3c50-433c-bd9c-be054324e1b1 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:24.124471 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1074/2115] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:28:24 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:28:25.087266 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94fa46-51b0-4a28-bccc-4395b90a7d26 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:25.088193 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94fa46-51b0-4a28-bccc-4395b90a7d26 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:25.088904 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94fa46-51b0-4a28-bccc-4395b90a7d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:25.129627 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-de94fa46-51b0-4a28-bccc-4395b90a7d26 None None] MFA Rules not processed for user `b84dde57ff8e47419866377a819bea21`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:28:25.132011 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-de94fa46-51b0-4a28-bccc-4395b90a7d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:25.141616 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:28:25.205595 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1042/2116] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Jul 18 06:28:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:28:25.260980 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c45e9b-3c50-433c-bd9c-be054324e1b1 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:25.262428 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-f4c45e9b-3c50-433c-bd9c-be054324e1b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:25.379492 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c45e9b-3c50-433c-bd9c-be054324e1b1 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:25.387332 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4baf5-c627-4c9b-9815-f18858e95673 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0681f5fbea6d415c9ad2f3afae256a5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:25.388060 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4baf5-c627-4c9b-9815-f18858e95673 service swift] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:25.388369 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4baf5-c627-4c9b-9815-f18858e95673 service swift] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:25.388655 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4baf5-c627-4c9b-9815-f18858e95673 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:25.398268 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe4baf5-c627-4c9b-9815-f18858e95673 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:25.399877 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe4baf5-c627-4c9b-9815-f18858e95673 service swift] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:25.408914 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1075/2117] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Jul 18 06:28:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 06:28:26.377597 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f22359d-2c5a-4f1c-8a63-9bd00b3703cf None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:26.378096 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f22359d-2c5a-4f1c-8a63-9bd00b3703cf None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:26.378222 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f22359d-2c5a-4f1c-8a63-9bd00b3703cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:26.398467 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-6f22359d-2c5a-4f1c-8a63-9bd00b3703cf None None] MFA Rules not processed for user `b84dde57ff8e47419866377a819bea21`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:28:26.399694 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-6f22359d-2c5a-4f1c-8a63-9bd00b3703cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:26.411376 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1043/2118] 10.222.0.40 () {58 vars in 967 bytes} [Mon Jul 18 06:28:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:28:26.424991 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4baf5-c627-4c9b-9815-f18858e95673 service swift] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:26.430609 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:28:26.436407 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4baf5-c627-4c9b-9815-f18858e95673 service swift] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:26.441558 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ce151-6c19-45d5-8e50-d6b7efb1464a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0681f5fbea6d415c9ad2f3afae256a5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:26.442210 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ce151-6c19-45d5-8e50-d6b7efb1464a service swift] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:26.442499 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ce151-6c19-45d5-8e50-d6b7efb1464a service swift] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:26.442812 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ce151-6c19-45d5-8e50-d6b7efb1464a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:26.452602 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248ce151-6c19-45d5-8e50-d6b7efb1464a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:26.454496 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248ce151-6c19-45d5-8e50-d6b7efb1464a service swift] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:26.463740 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1076/2119] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Jul 18 06:28:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 06:28:27.253089 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c85036-d1b0-4ece-8db8-515e14d1b9b1 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:27.253381 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c85036-d1b0-4ece-8db8-515e14d1b9b1 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:27.253831 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c85036-d1b0-4ece-8db8-515e14d1b9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:27.275811 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-42c85036-d1b0-4ece-8db8-515e14d1b9b1 None None] MFA Rules not processed for user `b84dde57ff8e47419866377a819bea21`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:28:27.277160 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-42c85036-d1b0-4ece-8db8-515e14d1b9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:27.285660 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1044/2120] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Jul 18 06:28:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:28:27.298422 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ce151-6c19-45d5-8e50-d6b7efb1464a service swift] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:27.309578 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ce151-6c19-45d5-8e50-d6b7efb1464a service swift] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:27.315519 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a8776-f47d-4f51-a8de-d4b806a2181e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0681f5fbea6d415c9ad2f3afae256a5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:27.317118 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a8776-f47d-4f51-a8de-d4b806a2181e service swift] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:27.317413 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a8776-f47d-4f51-a8de-d4b806a2181e service swift] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:27.317702 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a8776-f47d-4f51-a8de-d4b806a2181e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:27.327104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a8776-f47d-4f51-a8de-d4b806a2181e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:27.329366 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a8776-f47d-4f51-a8de-d4b806a2181e service swift] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:27.338617 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1077/2121] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Jul 18 06:28:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 06:28:28.083213 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a8b48-99f3-4086-948a-d3fb574ab8e3 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:28.083709 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a8b48-99f3-4086-948a-d3fb574ab8e3 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:28.084638 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a8b48-99f3-4086-948a-d3fb574ab8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:28.115780 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-dc9a8b48-99f3-4086-948a-d3fb574ab8e3 None None] MFA Rules not processed for user `b84dde57ff8e47419866377a819bea21`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:28:28.117637 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-dc9a8b48-99f3-4086-948a-d3fb574ab8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:28.130682 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1045/2122] 10.222.0.40 () {58 vars in 967 bytes} [Mon Jul 18 06:28:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:28:28.147873 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a8776-f47d-4f51-a8de-d4b806a2181e service swift] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:28.161151 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a8776-f47d-4f51-a8de-d4b806a2181e service swift] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:28.167648 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66082761-3bba-40e5-a86f-fa72898ce481 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0681f5fbea6d415c9ad2f3afae256a5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:28.168540 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66082761-3bba-40e5-a86f-fa72898ce481 service swift] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:28.168930 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66082761-3bba-40e5-a86f-fa72898ce481 service swift] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:28.169308 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66082761-3bba-40e5-a86f-fa72898ce481 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:28.185916 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66082761-3bba-40e5-a86f-fa72898ce481 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:28.190197 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66082761-3bba-40e5-a86f-fa72898ce481 service swift] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:28.206568 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1078/2123] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Jul 18 06:28:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 06:28:31.530745 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f82e3de-8433-44ce-99ed-695442c391e3 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:31.531352 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f82e3de-8433-44ce-99ed-695442c391e3 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:31.531352 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f82e3de-8433-44ce-99ed-695442c391e3 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:31.532791 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1046/2124] 10.222.0.40 () {54 vars in 921 bytes} [Mon Jul 18 06:28:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:28:31.568095 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf2305-090b-4413-9ae8-d329ea5bb0fb None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:31.568701 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf2305-090b-4413-9ae8-d329ea5bb0fb None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:31.568701 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf2305-090b-4413-9ae8-d329ea5bb0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:31.590582 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-aadf2305-090b-4413-9ae8-d329ea5bb0fb None None] MFA Rules not processed for user `a3560534427544cdabfd0ba702a0a165`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:28:31.591820 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-aadf2305-090b-4413-9ae8-d329ea5bb0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:31.665988 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1079/2125] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Jul 18 06:28:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:28:31.679926 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f82e3de-8433-44ce-99ed-695442c391e3 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:31.684146 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-2f82e3de-8433-44ce-99ed-695442c391e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:31.691670 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf2305-090b-4413-9ae8-d329ea5bb0fb None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:31.693417 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-aadf2305-090b-4413-9ae8-d329ea5bb0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:31.715277 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f82e3de-8433-44ce-99ed-695442c391e3 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:31.723235 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f4ed5-b32b-4e0c-8183-8bd381f2bc4c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3560534427544cdabfd0ba702a0a165', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:31.723903 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f4ed5-b32b-4e0c-8183-8bd381f2bc4c service placement] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:31.724190 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f4ed5-b32b-4e0c-8183-8bd381f2bc4c service placement] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:31.724575 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f4ed5-b32b-4e0c-8183-8bd381f2bc4c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:31.735227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f4ed5-b32b-4e0c-8183-8bd381f2bc4c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:31.738144 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f4ed5-b32b-4e0c-8183-8bd381f2bc4c service placement] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:31.751095 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1047/2126] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:28:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:28:31.803630 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf2305-090b-4413-9ae8-d329ea5bb0fb None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:31.809206 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3888eb-a5a0-4c73-a9cf-a741677ad977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:31.810308 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3888eb-a5a0-4c73-a9cf-a741677ad977 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:31.810656 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3888eb-a5a0-4c73-a9cf-a741677ad977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:31.811002 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3888eb-a5a0-4c73-a9cf-a741677ad977 admin admin] PATH_INFO: `/v3/users/788e2a26fe274360aed5498ef853e013` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:31.812479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3888eb-a5a0-4c73-a9cf-a741677ad977 admin admin] RBAC: Authorizing `identity:delete_user(user_id=788e2a26fe274360aed5498ef853e013)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:31.815014 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3888eb-a5a0-4c73-a9cf-a741677ad977 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:32.006787 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-af3888eb-a5a0-4c73-a9cf-a741677ad977 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '788e2a26fe274360aed5498ef853e013'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:32.014049 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-af3888eb-a5a0-4c73-a9cf-a741677ad977 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '788e2a26fe274360aed5498ef853e013'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:32.014819 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-af3888eb-a5a0-4c73-a9cf-a741677ad977 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '788e2a26fe274360aed5498ef853e013'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:32.015949 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-af3888eb-a5a0-4c73-a9cf-a741677ad977 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '788e2a26fe274360aed5498ef853e013'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:32.020751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:28:32.036307 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-af3888eb-a5a0-4c73-a9cf-a741677ad977 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '788e2a26fe274360aed5498ef853e013'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:32.124139 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1080/2127] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:28:31 2022] DELETE /identity/v3/users/788e2a26fe274360aed5498ef853e013 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:28:32.668533 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f4ed5-b32b-4e0c-8183-8bd381f2bc4c service placement] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:32.689510 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f4ed5-b32b-4e0c-8183-8bd381f2bc4c service placement] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:32.769267 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207564d6-0306-4a44-97b9-1b206a82686d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:32.770158 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207564d6-0306-4a44-97b9-1b206a82686d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:32.770446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207564d6-0306-4a44-97b9-1b206a82686d admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:32.770727 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207564d6-0306-4a44-97b9-1b206a82686d admin admin] PATH_INFO: `/v3/projects/8d18d6a16fe2472e86c4453f87eddc05` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:32.771993 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207564d6-0306-4a44-97b9-1b206a82686d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d18d6a16fe2472e86c4453f87eddc05)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:32.774132 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207564d6-0306-4a44-97b9-1b206a82686d admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:32.883495 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-207564d6-0306-4a44-97b9-1b206a82686d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d18d6a16fe2472e86c4453f87eddc05', 'request_id': 'req-207564d6-0306-4a44-97b9-1b206a82686d'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:32.889085 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-207564d6-0306-4a44-97b9-1b206a82686d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d18d6a16fe2472e86c4453f87eddc05', 'request_id': 'req-207564d6-0306-4a44-97b9-1b206a82686d'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:32.979772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-207564d6-0306-4a44-97b9-1b206a82686d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d18d6a16fe2472e86c4453f87eddc05', 'request_id': 'req-207564d6-0306-4a44-97b9-1b206a82686d'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:32.992625 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-207564d6-0306-4a44-97b9-1b206a82686d admin admin] The token cache is being invalidate because project 8d18d6a16fe2472e86c4453f87eddc05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:28:32.992923 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-207564d6-0306-4a44-97b9-1b206a82686d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:32.993585 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-207564d6-0306-4a44-97b9-1b206a82686d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:32.995385 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1048/2128] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:28:32 2022] DELETE /identity/v3/projects/8d18d6a16fe2472e86c4453f87eddc05 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:28:33.001171 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce7adb-2780-4877-8954-5c24c5ac26ad None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:33.001751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce7adb-2780-4877-8954-5c24c5ac26ad None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:33.001943 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce7adb-2780-4877-8954-5c24c5ac26ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:33.021132 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-15ce7adb-2780-4877-8954-5c24c5ac26ad None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:28:33.022727 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-15ce7adb-2780-4877-8954-5c24c5ac26ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:33.094654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1081/2129] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:28:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:28:33.102071 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1059f4-d247-49eb-8cc2-fa1ec05676c6 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:33.102390 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1059f4-d247-49eb-8cc2-fa1ec05676c6 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:33.102676 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1059f4-d247-49eb-8cc2-fa1ec05676c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:33.127983 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-3e1059f4-d247-49eb-8cc2-fa1ec05676c6 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:28:33.129495 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-3e1059f4-d247-49eb-8cc2-fa1ec05676c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:33.138051 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1049/2130] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:28:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:28:33.142620 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce7adb-2780-4877-8954-5c24c5ac26ad None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:33.153971 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce7adb-2780-4877-8954-5c24c5ac26ad None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:33.159636 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08bcbff-683c-406a-bdc6-75e47830eef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:33.160312 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08bcbff-683c-406a-bdc6-75e47830eef1 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:33.160638 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08bcbff-683c-406a-bdc6-75e47830eef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:33.161105 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08bcbff-683c-406a-bdc6-75e47830eef1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:33.161847 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08bcbff-683c-406a-bdc6-75e47830eef1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:28:33.162158 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08bcbff-683c-406a-bdc6-75e47830eef1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:33.163786 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08bcbff-683c-406a-bdc6-75e47830eef1 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:33.176863 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1082/2131] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:28:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:28:33.181166 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1059f4-d247-49eb-8cc2-fa1ec05676c6 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:33.190875 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1059f4-d247-49eb-8cc2-fa1ec05676c6 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:33.198630 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91503fe1-0a3c-436e-a4fe-f11c2f28d4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:33.199224 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91503fe1-0a3c-436e-a4fe-f11c2f28d4a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:33.199507 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91503fe1-0a3c-436e-a4fe-f11c2f28d4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:33.199783 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91503fe1-0a3c-436e-a4fe-f11c2f28d4a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:33.200497 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91503fe1-0a3c-436e-a4fe-f11c2f28d4a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:33.202463 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91503fe1-0a3c-436e-a4fe-f11c2f28d4a9 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:33.205922 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:28:33.299943 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1050/2132] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:28:33 2022] POST /identity/v3/projects => generated 387 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:28:33.303251 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08bcbff-683c-406a-bdc6-75e47830eef1 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:33.313528 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08bcbff-683c-406a-bdc6-75e47830eef1 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:33.371349 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8945e595-0f16-4054-933d-4a650d64527b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:33.372030 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945e595-0f16-4054-933d-4a650d64527b admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:33.372311 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945e595-0f16-4054-933d-4a650d64527b admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:33.372623 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945e595-0f16-4054-933d-4a650d64527b admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:33.373313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8945e595-0f16-4054-933d-4a650d64527b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:33.375437 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8945e595-0f16-4054-933d-4a650d64527b admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:33.498273 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1083/2133] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:28:33 2022] POST /identity/v3/users => generated 294 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:28:33.503411 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91503fe1-0a3c-436e-a4fe-f11c2f28d4a9 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:33.524504 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91503fe1-0a3c-436e-a4fe-f11c2f28d4a9 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:33.529735 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f1497-b45b-4bbc-b613-86a6e5c871ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:33.530414 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f1497-b45b-4bbc-b613-86a6e5c871ad admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:33.530739 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f1497-b45b-4bbc-b613-86a6e5c871ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:33.531093 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f1497-b45b-4bbc-b613-86a6e5c871ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:33.532666 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813f1497-b45b-4bbc-b613-86a6e5c871ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:33.534198 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813f1497-b45b-4bbc-b613-86a6e5c871ad admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:33.545189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1051/2134] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:28:33 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:28:33.550839 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8945e595-0f16-4054-933d-4a650d64527b admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:33.563492 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8945e595-0f16-4054-933d-4a650d64527b admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:33.571362 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3244612-43f0-4944-ac78-387759d96d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:33.572055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3244612-43f0-4944-ac78-387759d96d56 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:33.572344 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3244612-43f0-4944-ac78-387759d96d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:33.572659 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3244612-43f0-4944-ac78-387759d96d56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:33.573416 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3244612-43f0-4944-ac78-387759d96d56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:33.575048 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3244612-43f0-4944-ac78-387759d96d56 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:33.587224 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1084/2135] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:28:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 06:28:33.591786 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f1497-b45b-4bbc-b613-86a6e5c871ad admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:33.603427 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f1497-b45b-4bbc-b613-86a6e5c871ad admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:33.608683 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7dfb63-bb44-42b1-950c-dc302791e942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:33.609977 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7dfb63-bb44-42b1-950c-dc302791e942 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:33.610488 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7dfb63-bb44-42b1-950c-dc302791e942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:33.610910 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7dfb63-bb44-42b1-950c-dc302791e942 admin admin] PATH_INFO: `/v3/projects/31c9d72882974d3f96b4e5d3b4ec701d/users/936ea0959e924aac95cf474687e35c78/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:33.632656 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7dfb63-bb44-42b1-950c-dc302791e942 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31c9d72882974d3f96b4e5d3b4ec701d, user_id=936ea0959e924aac95cf474687e35c78, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:33.635340 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7dfb63-bb44-42b1-950c-dc302791e942 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:33.770354 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1052/2136] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:28:33 2022] PUT /identity/v3/projects/31c9d72882974d3f96b4e5d3b4ec701d/users/936ea0959e924aac95cf474687e35c78/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:28:33.776551 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b33d4f-8dff-4644-9482-cca9ef94706a None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:33.777066 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b33d4f-8dff-4644-9482-cca9ef94706a None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:33.777629 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b33d4f-8dff-4644-9482-cca9ef94706a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:33.804903 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-f8b33d4f-8dff-4644-9482-cca9ef94706a None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:28:33.806140 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-f8b33d4f-8dff-4644-9482-cca9ef94706a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:33.850576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:28:33.872353 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1085/2137] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:28:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:28:33.883354 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7dfb63-bb44-42b1-950c-dc302791e942 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:33.885149 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-1b7dfb63-bb44-42b1-950c-dc302791e942 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:33.951249 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7dfb63-bb44-42b1-950c-dc302791e942 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:33.955968 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3c2b8-15e8-4a02-92ad-b13f1f2c5dd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:33.957513 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3c2b8-15e8-4a02-92ad-b13f1f2c5dd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:33.957902 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3c2b8-15e8-4a02-92ad-b13f1f2c5dd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:33.958311 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3c2b8-15e8-4a02-92ad-b13f1f2c5dd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:33.968504 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b3c2b8-15e8-4a02-92ad-b13f1f2c5dd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:33.969923 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b3c2b8-15e8-4a02-92ad-b13f1f2c5dd2 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:33.980008 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1053/2138] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:28:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:28:36.753620 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b33d4f-8dff-4644-9482-cca9ef94706a None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:36.755209 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-f8b33d4f-8dff-4644-9482-cca9ef94706a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:36.768237 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b33d4f-8dff-4644-9482-cca9ef94706a None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:36.773196 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef9441-44eb-4518-8c42-945a5e78bfbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:36.774112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef9441-44eb-4518-8c42-945a5e78bfbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:36.774401 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef9441-44eb-4518-8c42-945a5e78bfbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:36.774685 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef9441-44eb-4518-8c42-945a5e78bfbe admin admin] PATH_INFO: `/v3/users/ba7eb26d3cf04471ac34ececfa569b43` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:36.775950 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef9441-44eb-4518-8c42-945a5e78bfbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba7eb26d3cf04471ac34ececfa569b43)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:36.778312 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef9441-44eb-4518-8c42-945a5e78bfbe admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:37.024143 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-35ef9441-44eb-4518-8c42-945a5e78bfbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba7eb26d3cf04471ac34ececfa569b43'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:37.045173 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-35ef9441-44eb-4518-8c42-945a5e78bfbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba7eb26d3cf04471ac34ececfa569b43'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:37.049942 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-35ef9441-44eb-4518-8c42-945a5e78bfbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba7eb26d3cf04471ac34ececfa569b43'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:37.049942 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-35ef9441-44eb-4518-8c42-945a5e78bfbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba7eb26d3cf04471ac34ececfa569b43'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:37.069011 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-35ef9441-44eb-4518-8c42-945a5e78bfbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba7eb26d3cf04471ac34ececfa569b43'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:37.193083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1086/2139] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:28:36 2022] DELETE /identity/v3/users/ba7eb26d3cf04471ac34ececfa569b43 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:28:37.200856 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc618d9-e403-4c55-8e9b-a6c1140e4e0e None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:37.201999 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc618d9-e403-4c55-8e9b-a6c1140e4e0e None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:37.202574 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc618d9-e403-4c55-8e9b-a6c1140e4e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:37.241243 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-7dc618d9-e403-4c55-8e9b-a6c1140e4e0e None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:28:37.242513 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7dc618d9-e403-4c55-8e9b-a6c1140e4e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:37.369765 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1054/2140] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:28:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:28:37.407879 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef9441-44eb-4518-8c42-945a5e78bfbe admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:37.409468 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-35ef9441-44eb-4518-8c42-945a5e78bfbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:37.439352 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc618d9-e403-4c55-8e9b-a6c1140e4e0e None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:37.440933 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7dc618d9-e403-4c55-8e9b-a6c1140e4e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:37.457716 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc618d9-e403-4c55-8e9b-a6c1140e4e0e None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:37.465624 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c93551-d2bf-462d-8d83-e2d4add39768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:37.466550 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c93551-d2bf-462d-8d83-e2d4add39768 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:37.466862 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c93551-d2bf-462d-8d83-e2d4add39768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:37.467125 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c93551-d2bf-462d-8d83-e2d4add39768 admin admin] PATH_INFO: `/v3/users/d2c771a986f4453a9f32242983c478d4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:37.468498 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c93551-d2bf-462d-8d83-e2d4add39768 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2c771a986f4453a9f32242983c478d4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:37.472002 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c93551-d2bf-462d-8d83-e2d4add39768 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:37.506621 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef9441-44eb-4518-8c42-945a5e78bfbe admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:37.520268 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c4c2f-a737-4c0b-87d4-197afb3ba065 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:37.521467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c4c2f-a737-4c0b-87d4-197afb3ba065 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:37.521979 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c4c2f-a737-4c0b-87d4-197afb3ba065 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:37.522514 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c4c2f-a737-4c0b-87d4-197afb3ba065 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:37.536315 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c4c2f-a737-4c0b-87d4-197afb3ba065 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:37.538784 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c4c2f-a737-4c0b-87d4-197afb3ba065 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:37.553103 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1087/2141] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:28:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:28:37.716005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-d2c93551-d2bf-462d-8d83-e2d4add39768 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2c771a986f4453a9f32242983c478d4'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:37.718030 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:28:37.734794 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-d2c93551-d2bf-462d-8d83-e2d4add39768 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2c771a986f4453a9f32242983c478d4'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:37.772995 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-d2c93551-d2bf-462d-8d83-e2d4add39768 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2c771a986f4453a9f32242983c478d4'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:37.774168 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-d2c93551-d2bf-462d-8d83-e2d4add39768 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2c771a986f4453a9f32242983c478d4'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:37.775241 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-d2c93551-d2bf-462d-8d83-e2d4add39768 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2c771a986f4453a9f32242983c478d4'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:37.868916 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1055/2142] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:28:37 2022] DELETE /identity/v3/users/d2c771a986f4453a9f32242983c478d4 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:28:37.873549 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4554f-43b1-4f40-930e-303c6c4e0a63 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:37.874910 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4554f-43b1-4f40-930e-303c6c4e0a63 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:37.875362 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4554f-43b1-4f40-930e-303c6c4e0a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:37.905089 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-15f4554f-43b1-4f40-930e-303c6c4e0a63 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:28:37.907259 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-15f4554f-43b1-4f40-930e-303c6c4e0a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:38.036883 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:28:38.057409 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1088/2143] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:28:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:28:38.076221 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c93551-d2bf-462d-8d83-e2d4add39768 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:38.076221 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-d2c93551-d2bf-462d-8d83-e2d4add39768 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:38.092172 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f4554f-43b1-4f40-930e-303c6c4e0a63 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:38.094878 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-15f4554f-43b1-4f40-930e-303c6c4e0a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:38.123416 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f4554f-43b1-4f40-930e-303c6c4e0a63 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:38.140100 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dca1fb-e8d4-4a0d-877c-e70520caae74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:38.142086 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dca1fb-e8d4-4a0d-877c-e70520caae74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:38.142446 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dca1fb-e8d4-4a0d-877c-e70520caae74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:38.142806 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dca1fb-e8d4-4a0d-877c-e70520caae74 admin admin] PATH_INFO: `/v3/projects/0c2f0aa6575940e7b01c039449ad0659` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:38.146641 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dca1fb-e8d4-4a0d-877c-e70520caae74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c2f0aa6575940e7b01c039449ad0659)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:38.151888 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dca1fb-e8d4-4a0d-877c-e70520caae74 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:38.202205 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c93551-d2bf-462d-8d83-e2d4add39768 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:38.210293 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4ecc8-e31f-4711-be36-e37c5caea966 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:38.211194 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4ecc8-e31f-4711-be36-e37c5caea966 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:38.212075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4ecc8-e31f-4711-be36-e37c5caea966 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:38.212647 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4ecc8-e31f-4711-be36-e37c5caea966 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:38.223647 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb4ecc8-e31f-4711-be36-e37c5caea966 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:38.226044 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb4ecc8-e31f-4711-be36-e37c5caea966 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:38.242919 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1056/2144] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:28:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 06:28:38.360857 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-04dca1fb-e8d4-4a0d-877c-e70520caae74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c2f0aa6575940e7b01c039449ad0659', 'request_id': 'req-04dca1fb-e8d4-4a0d-877c-e70520caae74'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:38.382788 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-04dca1fb-e8d4-4a0d-877c-e70520caae74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c2f0aa6575940e7b01c039449ad0659', 'request_id': 'req-04dca1fb-e8d4-4a0d-877c-e70520caae74'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:38.477781 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-04dca1fb-e8d4-4a0d-877c-e70520caae74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c2f0aa6575940e7b01c039449ad0659', 'request_id': 'req-04dca1fb-e8d4-4a0d-877c-e70520caae74'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:38.494610 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-04dca1fb-e8d4-4a0d-877c-e70520caae74 admin admin] The token cache is being invalidate because project 0c2f0aa6575940e7b01c039449ad0659 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:28:38.494887 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-04dca1fb-e8d4-4a0d-877c-e70520caae74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:38.496579 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-04dca1fb-e8d4-4a0d-877c-e70520caae74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:38.502510 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1089/2145] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:28:38 2022] DELETE /identity/v3/projects/0c2f0aa6575940e7b01c039449ad0659 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:28:38.573814 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4ecc8-e31f-4711-be36-e37c5caea966 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:38.575432 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-adb4ecc8-e31f-4711-be36-e37c5caea966 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:38.595792 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4ecc8-e31f-4711-be36-e37c5caea966 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:38.747144 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e015e3b-ebf0-48f9-98ec-f2f1e83f5b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:38.748074 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e015e3b-ebf0-48f9-98ec-f2f1e83f5b2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:38.748664 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e015e3b-ebf0-48f9-98ec-f2f1e83f5b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:38.748815 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e015e3b-ebf0-48f9-98ec-f2f1e83f5b2c admin admin] PATH_INFO: `/v3/users/4bea4bd0562a4df390a522e24bb9bd55` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:38.750438 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e015e3b-ebf0-48f9-98ec-f2f1e83f5b2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bea4bd0562a4df390a522e24bb9bd55)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:38.753579 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e015e3b-ebf0-48f9-98ec-f2f1e83f5b2c admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:39.005380 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6e015e3b-ebf0-48f9-98ec-f2f1e83f5b2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bea4bd0562a4df390a522e24bb9bd55'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:39.045782 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6e015e3b-ebf0-48f9-98ec-f2f1e83f5b2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bea4bd0562a4df390a522e24bb9bd55'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:39.087430 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:28:39.092560 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6e015e3b-ebf0-48f9-98ec-f2f1e83f5b2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bea4bd0562a4df390a522e24bb9bd55'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:39.093502 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6e015e3b-ebf0-48f9-98ec-f2f1e83f5b2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bea4bd0562a4df390a522e24bb9bd55'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:39.094964 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6e015e3b-ebf0-48f9-98ec-f2f1e83f5b2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bea4bd0562a4df390a522e24bb9bd55'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:39.239593 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1057/2146] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:28:38 2022] DELETE /identity/v3/users/4bea4bd0562a4df390a522e24bb9bd55 => generated 0 bytes in 662 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:28:39.240838 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f9e47-acfa-41f2-83df-1e5b0b0e0d40 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:39.241281 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f9e47-acfa-41f2-83df-1e5b0b0e0d40 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:39.241628 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f9e47-acfa-41f2-83df-1e5b0b0e0d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:39.261406 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-a57f9e47-acfa-41f2-83df-1e5b0b0e0d40 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:28:39.262748 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-a57f9e47-acfa-41f2-83df-1e5b0b0e0d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:39.342950 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1090/2147] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:28:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:28:39.361923 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e015e3b-ebf0-48f9-98ec-f2f1e83f5b2c admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:39.361923 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-6e015e3b-ebf0-48f9-98ec-f2f1e83f5b2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:39.376391 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f9e47-acfa-41f2-83df-1e5b0b0e0d40 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:39.379151 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-a57f9e47-acfa-41f2-83df-1e5b0b0e0d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:39.408102 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f9e47-acfa-41f2-83df-1e5b0b0e0d40 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:39.414998 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f1d40-46b7-4790-a350-f389c66541d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:39.420776 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f1d40-46b7-4790-a350-f389c66541d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:39.420776 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f1d40-46b7-4790-a350-f389c66541d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:39.420776 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f1d40-46b7-4790-a350-f389c66541d2 admin admin] PATH_INFO: `/v3/projects/ecb6ea27e82944059022e671cd651978` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:39.423875 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0f1d40-46b7-4790-a350-f389c66541d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecb6ea27e82944059022e671cd651978)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:39.426778 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0f1d40-46b7-4790-a350-f389c66541d2 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:39.475060 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e015e3b-ebf0-48f9-98ec-f2f1e83f5b2c admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:39.482471 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7d07f-fa2f-4ee2-b56e-2599fff40bbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:39.483190 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7d07f-fa2f-4ee2-b56e-2599fff40bbd service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:39.483484 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7d07f-fa2f-4ee2-b56e-2599fff40bbd service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:39.483756 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7d07f-fa2f-4ee2-b56e-2599fff40bbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:39.504961 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f7d07f-fa2f-4ee2-b56e-2599fff40bbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:39.506469 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f7d07f-fa2f-4ee2-b56e-2599fff40bbd service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:39.545869 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1058/2148] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:28:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:28:39.563659 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ce0f1d40-46b7-4790-a350-f389c66541d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecb6ea27e82944059022e671cd651978', 'request_id': 'req-ce0f1d40-46b7-4790-a350-f389c66541d2'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:39.579569 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ce0f1d40-46b7-4790-a350-f389c66541d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecb6ea27e82944059022e671cd651978', 'request_id': 'req-ce0f1d40-46b7-4790-a350-f389c66541d2'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:39.654801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ce0f1d40-46b7-4790-a350-f389c66541d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecb6ea27e82944059022e671cd651978', 'request_id': 'req-ce0f1d40-46b7-4790-a350-f389c66541d2'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:39.680175 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ce0f1d40-46b7-4790-a350-f389c66541d2 admin admin] The token cache is being invalidate because project ecb6ea27e82944059022e671cd651978 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:28:39.680696 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ce0f1d40-46b7-4790-a350-f389c66541d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:39.681586 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ce0f1d40-46b7-4790-a350-f389c66541d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:39.688986 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1091/2149] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:28:39 2022] DELETE /identity/v3/projects/ecb6ea27e82944059022e671cd651978 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:28:39.693389 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7d07f-fa2f-4ee2-b56e-2599fff40bbd service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:39.707914 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7d07f-fa2f-4ee2-b56e-2599fff40bbd service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:39.866100 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a430fb9-4cb2-4893-9f93-d67bcc8fc432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:39.866778 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a430fb9-4cb2-4893-9f93-d67bcc8fc432 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:39.867182 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a430fb9-4cb2-4893-9f93-d67bcc8fc432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:39.867450 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a430fb9-4cb2-4893-9f93-d67bcc8fc432 admin admin] PATH_INFO: `/v3/users/ab0730b770254e7b8d4af4c3fd26003e` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:39.869094 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a430fb9-4cb2-4893-9f93-d67bcc8fc432 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab0730b770254e7b8d4af4c3fd26003e)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:39.871120 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a430fb9-4cb2-4893-9f93-d67bcc8fc432 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:40.151032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-4a430fb9-4cb2-4893-9f93-d67bcc8fc432 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab0730b770254e7b8d4af4c3fd26003e'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:40.162266 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-4a430fb9-4cb2-4893-9f93-d67bcc8fc432 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab0730b770254e7b8d4af4c3fd26003e'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:40.181437 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-4a430fb9-4cb2-4893-9f93-d67bcc8fc432 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab0730b770254e7b8d4af4c3fd26003e'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:40.182340 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-4a430fb9-4cb2-4893-9f93-d67bcc8fc432 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab0730b770254e7b8d4af4c3fd26003e'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:40.183335 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-4a430fb9-4cb2-4893-9f93-d67bcc8fc432 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab0730b770254e7b8d4af4c3fd26003e'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:40.324199 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1059/2150] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:28:39 2022] DELETE /identity/v3/users/ab0730b770254e7b8d4af4c3fd26003e => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:28:40.510735 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f1d40-46b7-4790-a350-f389c66541d2 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:40.512291 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ce0f1d40-46b7-4790-a350-f389c66541d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:40.528452 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f1d40-46b7-4790-a350-f389c66541d2 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:40.535143 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:28:40.612245 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6084d3-0f52-4f87-b9e3-2ece79a72a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:40.613174 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6084d3-0f52-4f87-b9e3-2ece79a72a7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:40.613667 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6084d3-0f52-4f87-b9e3-2ece79a72a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:40.613947 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6084d3-0f52-4f87-b9e3-2ece79a72a7b admin admin] PATH_INFO: `/v3/projects/ddee550fc53841359ae9a46735298334` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:40.615411 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6084d3-0f52-4f87-b9e3-2ece79a72a7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddee550fc53841359ae9a46735298334)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:40.617406 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6084d3-0f52-4f87-b9e3-2ece79a72a7b admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:40.793135 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6d6084d3-0f52-4f87-b9e3-2ece79a72a7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddee550fc53841359ae9a46735298334', 'request_id': 'req-6d6084d3-0f52-4f87-b9e3-2ece79a72a7b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:40.814649 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6d6084d3-0f52-4f87-b9e3-2ece79a72a7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddee550fc53841359ae9a46735298334', 'request_id': 'req-6d6084d3-0f52-4f87-b9e3-2ece79a72a7b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:40.894932 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6d6084d3-0f52-4f87-b9e3-2ece79a72a7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddee550fc53841359ae9a46735298334', 'request_id': 'req-6d6084d3-0f52-4f87-b9e3-2ece79a72a7b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:40.915534 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6d6084d3-0f52-4f87-b9e3-2ece79a72a7b admin admin] The token cache is being invalidate because project ddee550fc53841359ae9a46735298334 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:28:40.915960 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6d6084d3-0f52-4f87-b9e3-2ece79a72a7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:40.917098 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6d6084d3-0f52-4f87-b9e3-2ece79a72a7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:40.920710 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1092/2151] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:28:40 2022] DELETE /identity/v3/projects/ddee550fc53841359ae9a46735298334 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:28:40.926659 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a430fb9-4cb2-4893-9f93-d67bcc8fc432 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:40.929367 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4a430fb9-4cb2-4893-9f93-d67bcc8fc432 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:40.956961 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a430fb9-4cb2-4893-9f93-d67bcc8fc432 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:40.968348 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:28:41.111956 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6084d3-0f52-4f87-b9e3-2ece79a72a7b admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:41.121832 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-6d6084d3-0f52-4f87-b9e3-2ece79a72a7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:41.142122 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad9e3f8-b8b6-4f4f-b604-51e6a0b17716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:41.149378 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9e3f8-b8b6-4f4f-b604-51e6a0b17716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:41.149674 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9e3f8-b8b6-4f4f-b604-51e6a0b17716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:41.151123 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9e3f8-b8b6-4f4f-b604-51e6a0b17716 admin admin] PATH_INFO: `/v3/users/96411fff4f144b70b46b21670cb6f9ac` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:41.155603 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad9e3f8-b8b6-4f4f-b604-51e6a0b17716 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96411fff4f144b70b46b21670cb6f9ac)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:41.155982 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad9e3f8-b8b6-4f4f-b604-51e6a0b17716 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:41.191761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6084d3-0f52-4f87-b9e3-2ece79a72a7b admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:41.214206 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e54968-512c-48b0-9597-a9c2d8ad327d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:41.215999 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e54968-512c-48b0-9597-a9c2d8ad327d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:41.219025 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e54968-512c-48b0-9597-a9c2d8ad327d admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:41.220620 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e54968-512c-48b0-9597-a9c2d8ad327d admin admin] PATH_INFO: `/v3/projects/e013d04313824ca798b86a0352d8afdd` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:41.226408 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e54968-512c-48b0-9597-a9c2d8ad327d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e013d04313824ca798b86a0352d8afdd)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:41.235063 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e54968-512c-48b0-9597-a9c2d8ad327d admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:41.520647 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-aad9e3f8-b8b6-4f4f-b604-51e6a0b17716 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96411fff4f144b70b46b21670cb6f9ac'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:41.528151 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-02e54968-512c-48b0-9597-a9c2d8ad327d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e013d04313824ca798b86a0352d8afdd', 'request_id': 'req-02e54968-512c-48b0-9597-a9c2d8ad327d'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:41.531398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-aad9e3f8-b8b6-4f4f-b604-51e6a0b17716 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96411fff4f144b70b46b21670cb6f9ac'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:41.543763 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-02e54968-512c-48b0-9597-a9c2d8ad327d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e013d04313824ca798b86a0352d8afdd', 'request_id': 'req-02e54968-512c-48b0-9597-a9c2d8ad327d'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:41.548389 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-aad9e3f8-b8b6-4f4f-b604-51e6a0b17716 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96411fff4f144b70b46b21670cb6f9ac'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:41.549489 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-aad9e3f8-b8b6-4f4f-b604-51e6a0b17716 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96411fff4f144b70b46b21670cb6f9ac'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:41.550372 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-aad9e3f8-b8b6-4f4f-b604-51e6a0b17716 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96411fff4f144b70b46b21670cb6f9ac'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:41.621246 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-02e54968-512c-48b0-9597-a9c2d8ad327d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e013d04313824ca798b86a0352d8afdd', 'request_id': 'req-02e54968-512c-48b0-9597-a9c2d8ad327d'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:41.637027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-02e54968-512c-48b0-9597-a9c2d8ad327d admin admin] The token cache is being invalidate because project e013d04313824ca798b86a0352d8afdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:28:41.637297 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-02e54968-512c-48b0-9597-a9c2d8ad327d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:41.638462 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-02e54968-512c-48b0-9597-a9c2d8ad327d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:41.640083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1093/2152] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:28:41 2022] DELETE /identity/v3/projects/e013d04313824ca798b86a0352d8afdd => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:28:41.644285 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ae6fe-27e7-4578-bd94-b3a857b26924 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:41.644285 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ae6fe-27e7-4578-bd94-b3a857b26924 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:41.644285 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ae6fe-27e7-4578-bd94-b3a857b26924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:41.680190 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-4e1ae6fe-27e7-4578-bd94-b3a857b26924 None None] MFA Rules not processed for user `936ea0959e924aac95cf474687e35c78`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:28:41.681720 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4e1ae6fe-27e7-4578-bd94-b3a857b26924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:41.721815 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1060/2153] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:28:40 2022] DELETE /identity/v3/users/96411fff4f144b70b46b21670cb6f9ac => generated 0 bytes in 797 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:28:41.723544 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b28bcd-b606-4239-ba61-bd9cb0d4013c None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:41.723851 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b28bcd-b606-4239-ba61-bd9cb0d4013c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:41.724130 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b28bcd-b606-4239-ba61-bd9cb0d4013c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:41.762638 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-05b28bcd-b606-4239-ba61-bd9cb0d4013c None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:28:41.764152 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-05b28bcd-b606-4239-ba61-bd9cb0d4013c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:41.858249 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1094/2154] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:28:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:28:41.877858 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ae6fe-27e7-4578-bd94-b3a857b26924 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:41.879378 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4e1ae6fe-27e7-4578-bd94-b3a857b26924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:41.904668 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1061/2155] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:28:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:28:41.915815 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c150b41-f717-4c7c-989c-722831260d9d None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:41.916174 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c150b41-f717-4c7c-989c-722831260d9d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:41.916487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c150b41-f717-4c7c-989c-722831260d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:41.947103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-4c150b41-f717-4c7c-989c-722831260d9d None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:28:41.948533 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4c150b41-f717-4c7c-989c-722831260d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:41.966326 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1062/2156] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:28:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:28:41.968135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c150b41-f717-4c7c-989c-722831260d9d None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:41.980637 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c150b41-f717-4c7c-989c-722831260d9d None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:41.985332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90de1d0-d712-4db6-a2d2-cea091837bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:41.986304 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90de1d0-d712-4db6-a2d2-cea091837bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:41.986710 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90de1d0-d712-4db6-a2d2-cea091837bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:41.987001 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90de1d0-d712-4db6-a2d2-cea091837bc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:41.988324 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90de1d0-d712-4db6-a2d2-cea091837bc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:28:41.988763 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90de1d0-d712-4db6-a2d2-cea091837bc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:41.990777 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90de1d0-d712-4db6-a2d2-cea091837bc3 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:42.006627 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1063/2157] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:28:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:28:42.012436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90de1d0-d712-4db6-a2d2-cea091837bc3 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:42.019081 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ae6fe-27e7-4578-bd94-b3a857b26924 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:42.026409 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90de1d0-d712-4db6-a2d2-cea091837bc3 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:42.032782 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b945d8-1021-4a35-9ef4-abf37ef3d189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:42.034202 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b945d8-1021-4a35-9ef4-abf37ef3d189 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:42.034558 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b945d8-1021-4a35-9ef4-abf37ef3d189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:42.034946 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b945d8-1021-4a35-9ef4-abf37ef3d189 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:42.038013 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b945d8-1021-4a35-9ef4-abf37ef3d189 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:42.038923 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b945d8-1021-4a35-9ef4-abf37ef3d189 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:42.109645 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868b2d93-1a8c-4cc3-8cce-cc48e18345b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:42.111043 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b2d93-1a8c-4cc3-8cce-cc48e18345b5 service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:42.111394 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b2d93-1a8c-4cc3-8cce-cc48e18345b5 service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:42.111688 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b2d93-1a8c-4cc3-8cce-cc48e18345b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:42.178467 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1064/2158] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:28:42 2022] POST /identity/v3/projects => generated 365 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:28:42.182983 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b945d8-1021-4a35-9ef4-abf37ef3d189 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:42.223469 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868b2d93-1a8c-4cc3-8cce-cc48e18345b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:42.225634 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868b2d93-1a8c-4cc3-8cce-cc48e18345b5 service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:42.238925 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1095/2159] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:28:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:28:42.277770 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b945d8-1021-4a35-9ef4-abf37ef3d189 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:42.285367 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06a135-f168-43bc-91d0-77c68aeb1584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:42.286715 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06a135-f168-43bc-91d0-77c68aeb1584 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:42.287297 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06a135-f168-43bc-91d0-77c68aeb1584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:42.287959 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06a135-f168-43bc-91d0-77c68aeb1584 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:42.289392 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06a135-f168-43bc-91d0-77c68aeb1584 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:42.292314 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06a135-f168-43bc-91d0-77c68aeb1584 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:42.371230 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868b2d93-1a8c-4cc3-8cce-cc48e18345b5 service nova] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:42.372790 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-868b2d93-1a8c-4cc3-8cce-cc48e18345b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:42.390207 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868b2d93-1a8c-4cc3-8cce-cc48e18345b5 service nova] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:42.403408 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f48322-26b2-4b57-aa0b-5ba75e3f3f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:42.403408 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f48322-26b2-4b57-aa0b-5ba75e3f3f29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:42.404282 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f48322-26b2-4b57-aa0b-5ba75e3f3f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:42.404282 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f48322-26b2-4b57-aa0b-5ba75e3f3f29 admin admin] PATH_INFO: `/v3/projects/4cb86cac3344490d8df5fdd964a2df27` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:42.411612 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1065/2160] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:28:42 2022] POST /identity/v3/users => generated 283 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:28:42.411888 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f48322-26b2-4b57-aa0b-5ba75e3f3f29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cb86cac3344490d8df5fdd964a2df27)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:42.412818 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06a135-f168-43bc-91d0-77c68aeb1584 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:42.414577 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f48322-26b2-4b57-aa0b-5ba75e3f3f29 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:42.430711 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06a135-f168-43bc-91d0-77c68aeb1584 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:42.437661 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b13cb-6f27-4ee6-9fae-35d8a2860e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:42.438359 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b13cb-6f27-4ee6-9fae-35d8a2860e02 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:42.438649 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b13cb-6f27-4ee6-9fae-35d8a2860e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:42.438934 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b13cb-6f27-4ee6-9fae-35d8a2860e02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:42.439684 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8b13cb-6f27-4ee6-9fae-35d8a2860e02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:42.441364 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8b13cb-6f27-4ee6-9fae-35d8a2860e02 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:42.452339 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1066/2161] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:28:42.456587 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b13cb-6f27-4ee6-9fae-35d8a2860e02 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:42.467461 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b13cb-6f27-4ee6-9fae-35d8a2860e02 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:42.473173 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40481061-3737-41dc-befa-14dc290c0e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:42.473800 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40481061-3737-41dc-befa-14dc290c0e20 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:42.474087 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40481061-3737-41dc-befa-14dc290c0e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:42.474368 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40481061-3737-41dc-befa-14dc290c0e20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:42.475114 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40481061-3737-41dc-befa-14dc290c0e20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:42.476857 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40481061-3737-41dc-befa-14dc290c0e20 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:42.489568 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1067/2162] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 06:28:42.493388 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40481061-3737-41dc-befa-14dc290c0e20 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:42.504965 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40481061-3737-41dc-befa-14dc290c0e20 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:42.513923 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c4acdd-f1af-4491-8e3e-30189dd550b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:42.513923 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4acdd-f1af-4491-8e3e-30189dd550b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:42.513923 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4acdd-f1af-4491-8e3e-30189dd550b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:42.513923 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4acdd-f1af-4491-8e3e-30189dd550b1 admin admin] PATH_INFO: `/v3/projects/367b98c53f844498bebdc4be8392d92a/users/400a779e2f8f4c14b873fa49fd3a8cfe/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:42.530061 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c4acdd-f1af-4491-8e3e-30189dd550b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=367b98c53f844498bebdc4be8392d92a, user_id=400a779e2f8f4c14b873fa49fd3a8cfe, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:42.534500 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c4acdd-f1af-4491-8e3e-30189dd550b1 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:42.638974 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1068/2163] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:28:42 2022] PUT /identity/v3/projects/367b98c53f844498bebdc4be8392d92a/users/400a779e2f8f4c14b873fa49fd3a8cfe/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:28:42.659118 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39fce15-5e92-4c7d-b459-ca36527d21a8 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:42.659673 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39fce15-5e92-4c7d-b459-ca36527d21a8 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:42.660111 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39fce15-5e92-4c7d-b459-ca36527d21a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:42.673830 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:28:42.678836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-74f48322-26b2-4b57-aa0b-5ba75e3f3f29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cb86cac3344490d8df5fdd964a2df27', 'request_id': 'req-74f48322-26b2-4b57-aa0b-5ba75e3f3f29'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:42.693427 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-74f48322-26b2-4b57-aa0b-5ba75e3f3f29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cb86cac3344490d8df5fdd964a2df27', 'request_id': 'req-74f48322-26b2-4b57-aa0b-5ba75e3f3f29'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:42.717652 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-a39fce15-5e92-4c7d-b459-ca36527d21a8 None None] MFA Rules not processed for user `400a779e2f8f4c14b873fa49fd3a8cfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:28:42.720100 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a39fce15-5e92-4c7d-b459-ca36527d21a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:42.802326 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1069/2164] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:28:42 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:28:42.803487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39fce15-5e92-4c7d-b459-ca36527d21a8 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:42.826462 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-74f48322-26b2-4b57-aa0b-5ba75e3f3f29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cb86cac3344490d8df5fdd964a2df27', 'request_id': 'req-74f48322-26b2-4b57-aa0b-5ba75e3f3f29'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:42.839543 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-74f48322-26b2-4b57-aa0b-5ba75e3f3f29 admin admin] The token cache is being invalidate because project 4cb86cac3344490d8df5fdd964a2df27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:28:42.839845 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-74f48322-26b2-4b57-aa0b-5ba75e3f3f29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:42.842029 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-74f48322-26b2-4b57-aa0b-5ba75e3f3f29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:42.842481 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1096/2165] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:28:42 2022] DELETE /identity/v3/projects/4cb86cac3344490d8df5fdd964a2df27 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:28:42.848527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e688697a-4115-4b78-9021-f7cffdc297c0 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:42.848819 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e688697a-4115-4b78-9021-f7cffdc297c0 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:42.849107 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e688697a-4115-4b78-9021-f7cffdc297c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:42.869818 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-e688697a-4115-4b78-9021-f7cffdc297c0 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:28:42.870035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39fce15-5e92-4c7d-b459-ca36527d21a8 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:42.871278 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e688697a-4115-4b78-9021-f7cffdc297c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:42.877345 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9783a3c-6762-466a-a0ec-c7550f887ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:42.878295 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9783a3c-6762-466a-a0ec-c7550f887ecb admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:42.878569 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9783a3c-6762-466a-a0ec-c7550f887ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:42.878811 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9783a3c-6762-466a-a0ec-c7550f887ecb admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:42.879597 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9783a3c-6762-466a-a0ec-c7550f887ecb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:42.880881 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1097/2166] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:28:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:28:42.881924 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9783a3c-6762-466a-a0ec-c7550f887ecb admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:42.889281 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094d889-8c1a-44ab-9d26-f292cee68503 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:42.889668 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094d889-8c1a-44ab-9d26-f292cee68503 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:42.889893 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094d889-8c1a-44ab-9d26-f292cee68503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:42.912917 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-6094d889-8c1a-44ab-9d26-f292cee68503 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:28:42.919037 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-6094d889-8c1a-44ab-9d26-f292cee68503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:42.931875 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1098/2167] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:28:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:28:42.937883 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6094d889-8c1a-44ab-9d26-f292cee68503 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:42.951219 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6094d889-8c1a-44ab-9d26-f292cee68503 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:42.959995 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c7fd6-83fc-47ff-b31b-e1f8f315c586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:42.960917 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c7fd6-83fc-47ff-b31b-e1f8f315c586 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:42.961273 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c7fd6-83fc-47ff-b31b-e1f8f315c586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:42.961692 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c7fd6-83fc-47ff-b31b-e1f8f315c586 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:42.962655 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8c7fd6-83fc-47ff-b31b-e1f8f315c586 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:28:42.963072 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8c7fd6-83fc-47ff-b31b-e1f8f315c586 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:42.964933 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8c7fd6-83fc-47ff-b31b-e1f8f315c586 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:42.986126 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1099/2168] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:28:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:28:42.990778 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c7fd6-83fc-47ff-b31b-e1f8f315c586 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:43.008166 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c7fd6-83fc-47ff-b31b-e1f8f315c586 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:43.013316 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240d5b6-9c0b-4c7d-9afd-774fa2d5bb5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:43.014282 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240d5b6-9c0b-4c7d-9afd-774fa2d5bb5e admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:43.014455 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240d5b6-9c0b-4c7d-9afd-774fa2d5bb5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:43.014763 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240d5b6-9c0b-4c7d-9afd-774fa2d5bb5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:43.016058 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9240d5b6-9c0b-4c7d-9afd-774fa2d5bb5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:43.018981 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9240d5b6-9c0b-4c7d-9afd-774fa2d5bb5e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:43.048881 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1070/2169] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:28:42 2022] POST /identity/v3/projects => generated 365 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:28:43.051705 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9783a3c-6762-466a-a0ec-c7550f887ecb admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:43.053572 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f9783a3c-6762-466a-a0ec-c7550f887ecb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:43.139181 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1100/2170] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:28:42 2022] POST /identity/v3/projects => generated 357 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:28:43.143852 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240d5b6-9c0b-4c7d-9afd-774fa2d5bb5e admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:43.168152 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9783a3c-6762-466a-a0ec-c7550f887ecb admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:43.215987 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:28:43.244952 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525a686a-a2f6-49f3-951b-a1d2860da442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:43.245654 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a686a-a2f6-49f3-951b-a1d2860da442 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:43.245947 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a686a-a2f6-49f3-951b-a1d2860da442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:43.246230 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a686a-a2f6-49f3-951b-a1d2860da442 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:43.247118 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525a686a-a2f6-49f3-951b-a1d2860da442 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:43.250552 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525a686a-a2f6-49f3-951b-a1d2860da442 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:43.309852 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240d5b6-9c0b-4c7d-9afd-774fa2d5bb5e admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:43.318571 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825485c-5073-46fe-a2f0-7308224dd0fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:43.319678 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825485c-5073-46fe-a2f0-7308224dd0fa admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:43.319986 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825485c-5073-46fe-a2f0-7308224dd0fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:43.320709 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825485c-5073-46fe-a2f0-7308224dd0fa admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:43.321831 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a825485c-5073-46fe-a2f0-7308224dd0fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:43.324026 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a825485c-5073-46fe-a2f0-7308224dd0fa admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:43.437293 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1071/2171] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:28:43 2022] POST /identity/v3/users => generated 289 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:28:43.439863 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525a686a-a2f6-49f3-951b-a1d2860da442 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:43.458788 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525a686a-a2f6-49f3-951b-a1d2860da442 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:43.467151 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc5b01-e25c-44ac-b025-58f1f1769de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:43.470502 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc5b01-e25c-44ac-b025-58f1f1769de5 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:43.470502 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc5b01-e25c-44ac-b025-58f1f1769de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:43.470502 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc5b01-e25c-44ac-b025-58f1f1769de5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:43.470974 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1101/2172] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:28:43 2022] POST /identity/v3/users => generated 279 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:28:43.471118 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacc5b01-e25c-44ac-b025-58f1f1769de5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:43.474523 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacc5b01-e25c-44ac-b025-58f1f1769de5 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:43.478773 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825485c-5073-46fe-a2f0-7308224dd0fa admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:43.496903 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1072/2173] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:28:43.509205 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825485c-5073-46fe-a2f0-7308224dd0fa admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:43.510875 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc5b01-e25c-44ac-b025-58f1f1769de5 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:43.519597 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac850c39-3fc7-481b-bcf5-487e7ce97941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:43.521198 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac850c39-3fc7-481b-bcf5-487e7ce97941 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:43.521798 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac850c39-3fc7-481b-bcf5-487e7ce97941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:43.522408 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac850c39-3fc7-481b-bcf5-487e7ce97941 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:43.523848 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac850c39-3fc7-481b-bcf5-487e7ce97941 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:43.527502 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac850c39-3fc7-481b-bcf5-487e7ce97941 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:43.534387 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc5b01-e25c-44ac-b025-58f1f1769de5 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:43.546103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd18c6-dfae-4148-96d3-2f728e9efcf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:43.549041 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd18c6-dfae-4148-96d3-2f728e9efcf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:43.549795 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd18c6-dfae-4148-96d3-2f728e9efcf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:43.551408 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd18c6-dfae-4148-96d3-2f728e9efcf6 admin admin] PATH_INFO: `/v3/projects/c00d53c8ca3741c7b67d2234c24915e9/users/3ab4966dba1b4fc6a95129d5ebb033fb/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:43.560002 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1102/2174] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:28:43.565056 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac850c39-3fc7-481b-bcf5-487e7ce97941 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:43.582495 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac850c39-3fc7-481b-bcf5-487e7ce97941 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:43.591032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd18c6-dfae-4148-96d3-2f728e9efcf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c00d53c8ca3741c7b67d2234c24915e9, user_id=3ab4966dba1b4fc6a95129d5ebb033fb, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:43.598350 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd18c6-dfae-4148-96d3-2f728e9efcf6 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:43.612620 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66489911-d395-4157-a111-c1a440ec4a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:43.613928 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66489911-d395-4157-a111-c1a440ec4a2c admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:43.614476 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66489911-d395-4157-a111-c1a440ec4a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:43.614974 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66489911-d395-4157-a111-c1a440ec4a2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:43.616362 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66489911-d395-4157-a111-c1a440ec4a2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:43.619806 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66489911-d395-4157-a111-c1a440ec4a2c admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:43.646795 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1103/2175] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:28:43.654851 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66489911-d395-4157-a111-c1a440ec4a2c admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:43.684206 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66489911-d395-4157-a111-c1a440ec4a2c admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:43.695672 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7406b171-852a-4566-96df-939f6f1e228a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:43.697730 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7406b171-852a-4566-96df-939f6f1e228a admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:43.698355 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7406b171-852a-4566-96df-939f6f1e228a admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:43.698841 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7406b171-852a-4566-96df-939f6f1e228a admin admin] PATH_INFO: `/v3/projects/569a0a23d3564dc2937967da46f1299e/users/ef02ea3b60ca4466924379e36d09df0e/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:43.731422 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1073/2176] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:28:43 2022] PUT /identity/v3/projects/c00d53c8ca3741c7b67d2234c24915e9/users/3ab4966dba1b4fc6a95129d5ebb033fb/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:28:43.736886 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd18c6-dfae-4148-96d3-2f728e9efcf6 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:43.800483 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7406b171-852a-4566-96df-939f6f1e228a admin admin] RBAC: Authorizing `identity:create_grant(project_id=569a0a23d3564dc2937967da46f1299e, user_id=ef02ea3b60ca4466924379e36d09df0e, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:43.809527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7406b171-852a-4566-96df-939f6f1e228a admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:43.946595 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1104/2177] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:28:43 2022] PUT /identity/v3/projects/569a0a23d3564dc2937967da46f1299e/users/ef02ea3b60ca4466924379e36d09df0e/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:28:43.957643 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5984e5-ef24-43ee-942b-64a6404e900c None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:43.957643 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5984e5-ef24-43ee-942b-64a6404e900c None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:43.957643 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5984e5-ef24-43ee-942b-64a6404e900c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:43.987967 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd18c6-dfae-4148-96d3-2f728e9efcf6 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:44.015076 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-3d5984e5-ef24-43ee-942b-64a6404e900c None None] MFA Rules not processed for user `ef02ea3b60ca4466924379e36d09df0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:28:44.018624 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-3d5984e5-ef24-43ee-942b-64a6404e900c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:44.142626 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf328ad-770c-4764-844f-8651cdf40db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:44.145784 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf328ad-770c-4764-844f-8651cdf40db7 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:44.146385 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf328ad-770c-4764-844f-8651cdf40db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:44.148694 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf328ad-770c-4764-844f-8651cdf40db7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:44.155535 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf328ad-770c-4764-844f-8651cdf40db7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:44.182325 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf328ad-770c-4764-844f-8651cdf40db7 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:44.212456 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1074/2178] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 472 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:28:44.214115 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf328ad-770c-4764-844f-8651cdf40db7 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:44.226656 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1105/2179] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:28:43 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:28:44.234496 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5984e5-ef24-43ee-942b-64a6404e900c None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:44.256716 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5984e5-ef24-43ee-942b-64a6404e900c None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:44.268939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f110fc-8b51-4a96-b2b3-90041d6e0f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:44.269626 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f110fc-8b51-4a96-b2b3-90041d6e0f44 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:44.269914 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f110fc-8b51-4a96-b2b3-90041d6e0f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:44.270194 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f110fc-8b51-4a96-b2b3-90041d6e0f44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:44.274668 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f110fc-8b51-4a96-b2b3-90041d6e0f44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:44.277743 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f110fc-8b51-4a96-b2b3-90041d6e0f44 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:44.283950 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf328ad-770c-4764-844f-8651cdf40db7 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:44.314740 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20745a3b-813f-451d-989d-e6a77e8dbfa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:44.340938 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20745a3b-813f-451d-989d-e6a77e8dbfa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:44.340938 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20745a3b-813f-451d-989d-e6a77e8dbfa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:44.340938 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20745a3b-813f-451d-989d-e6a77e8dbfa6 admin admin] PATH_INFO: `/v3/projects/c00d53c8ca3741c7b67d2234c24915e9/users/3ab4966dba1b4fc6a95129d5ebb033fb/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:44.342404 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20745a3b-813f-451d-989d-e6a77e8dbfa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c00d53c8ca3741c7b67d2234c24915e9, user_id=3ab4966dba1b4fc6a95129d5ebb033fb, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:44.346164 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20745a3b-813f-451d-989d-e6a77e8dbfa6 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:44.389484 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1106/2180] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:28:44 2022] POST /identity/v3/projects => generated 357 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:28:44.395081 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f110fc-8b51-4a96-b2b3-90041d6e0f44 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:44.424840 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1075/2181] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:28:44 2022] PUT /identity/v3/projects/c00d53c8ca3741c7b67d2234c24915e9/users/3ab4966dba1b4fc6a95129d5ebb033fb/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:28:44.436231 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d67d08-e789-4b11-912d-3da7aef204b6 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:44.436528 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d67d08-e789-4b11-912d-3da7aef204b6 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:44.436755 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d67d08-e789-4b11-912d-3da7aef204b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:44.545539 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-54d67d08-e789-4b11-912d-3da7aef204b6 None None] MFA Rules not processed for user `3ab4966dba1b4fc6a95129d5ebb033fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:28:44.548182 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-54d67d08-e789-4b11-912d-3da7aef204b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:44.635065 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f110fc-8b51-4a96-b2b3-90041d6e0f44 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:44.773513 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1076/2182] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:28:44 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:28:44.781060 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d67d08-e789-4b11-912d-3da7aef204b6 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:44.784456 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-54d67d08-e789-4b11-912d-3da7aef204b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:44.869149 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d35dd2c-75e2-4537-b1df-7fb6e5f982c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:44.869964 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35dd2c-75e2-4537-b1df-7fb6e5f982c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:44.870260 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35dd2c-75e2-4537-b1df-7fb6e5f982c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:44.870546 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35dd2c-75e2-4537-b1df-7fb6e5f982c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:44.871320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d35dd2c-75e2-4537-b1df-7fb6e5f982c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:44.873421 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d35dd2c-75e2-4537-b1df-7fb6e5f982c4 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:44.892077 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d67d08-e789-4b11-912d-3da7aef204b6 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:44.897020 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea8849-0b40-498d-9eec-5cbc2c72c0c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:44.897776 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea8849-0b40-498d-9eec-5cbc2c72c0c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:44.898131 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea8849-0b40-498d-9eec-5cbc2c72c0c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:44.898487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea8849-0b40-498d-9eec-5cbc2c72c0c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:44.916537 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea8849-0b40-498d-9eec-5cbc2c72c0c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:44.918116 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea8849-0b40-498d-9eec-5cbc2c72c0c6 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:44.929388 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1077/2183] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:28:44 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:28:44.969134 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1107/2184] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:28:44 2022] POST /identity/v3/users => generated 285 bytes in 569 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:28:44.971432 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea8849-0b40-498d-9eec-5cbc2c72c0c6 service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:45.018760 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea8849-0b40-498d-9eec-5cbc2c72c0c6 service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:45.034988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1b2965-820d-435e-a98d-028ae8b22200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:45.035682 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b2965-820d-435e-a98d-028ae8b22200 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:45.035967 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b2965-820d-435e-a98d-028ae8b22200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:45.036254 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b2965-820d-435e-a98d-028ae8b22200 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:45.037064 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1b2965-820d-435e-a98d-028ae8b22200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:45.044089 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1b2965-820d-435e-a98d-028ae8b22200 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:45.055494 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:28:45.067515 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1078/2185] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:28:45.071370 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d35dd2c-75e2-4537-b1df-7fb6e5f982c4 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:45.118262 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d35dd2c-75e2-4537-b1df-7fb6e5f982c4 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:45.145113 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd09a022-d0a9-4e0e-bd96-a4810f91ef17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:45.146048 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09a022-d0a9-4e0e-bd96-a4810f91ef17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:45.146520 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09a022-d0a9-4e0e-bd96-a4810f91ef17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:45.146944 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09a022-d0a9-4e0e-bd96-a4810f91ef17 admin admin] PATH_INFO: `/v3/projects/2ffa0c4279a84e0085384a8f72c1fdc8/users/cfa8d8d688bd4560a66df89bd5ec20dd/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:45.198928 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd09a022-d0a9-4e0e-bd96-a4810f91ef17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ffa0c4279a84e0085384a8f72c1fdc8, user_id=cfa8d8d688bd4560a66df89bd5ec20dd, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:45.201824 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd09a022-d0a9-4e0e-bd96-a4810f91ef17 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:45.283361 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1b2965-820d-435e-a98d-028ae8b22200 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:45.286376 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4a1b2965-820d-435e-a98d-028ae8b22200 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:45.292545 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1108/2186] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:28:45 2022] PUT /identity/v3/projects/2ffa0c4279a84e0085384a8f72c1fdc8/users/cfa8d8d688bd4560a66df89bd5ec20dd/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:28:45.297501 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd09a022-d0a9-4e0e-bd96-a4810f91ef17 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:45.307287 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1b2965-820d-435e-a98d-028ae8b22200 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:45.370708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:28:45.405092 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd09a022-d0a9-4e0e-bd96-a4810f91ef17 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:45.412067 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57372b1e-aa51-4447-964f-b510bf2b78e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:45.412067 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57372b1e-aa51-4447-964f-b510bf2b78e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:45.412954 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57372b1e-aa51-4447-964f-b510bf2b78e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:45.413453 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57372b1e-aa51-4447-964f-b510bf2b78e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:45.414503 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57372b1e-aa51-4447-964f-b510bf2b78e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:45.416445 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57372b1e-aa51-4447-964f-b510bf2b78e6 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:45.418188 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60533109-4f9c-4fbd-bbec-68507f747948 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:45.419461 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60533109-4f9c-4fbd-bbec-68507f747948 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:45.419953 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60533109-4f9c-4fbd-bbec-68507f747948 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:45.420386 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60533109-4f9c-4fbd-bbec-68507f747948 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:45.423373 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-60533109-4f9c-4fbd-bbec-68507f747948 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:45.432585 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1109/2187] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:28:45 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:28:45.438633 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57372b1e-aa51-4447-964f-b510bf2b78e6 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:45.475702 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57372b1e-aa51-4447-964f-b510bf2b78e6 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:45.494003 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d64027-118a-405b-b6c5-3b931492a713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:45.495039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d64027-118a-405b-b6c5-3b931492a713 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:45.496150 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d64027-118a-405b-b6c5-3b931492a713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:45.496150 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d64027-118a-405b-b6c5-3b931492a713 admin admin] PATH_INFO: `/v3/projects/2ffa0c4279a84e0085384a8f72c1fdc8/users/cfa8d8d688bd4560a66df89bd5ec20dd/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:45.510051 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d64027-118a-405b-b6c5-3b931492a713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ffa0c4279a84e0085384a8f72c1fdc8, user_id=cfa8d8d688bd4560a66df89bd5ec20dd, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:45.522377 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d64027-118a-405b-b6c5-3b931492a713 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:45.535441 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60533109-4f9c-4fbd-bbec-68507f747948 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:45.537602 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60533109-4f9c-4fbd-bbec-68507f747948 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:45.561687 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1110/2188] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:28:45 2022] PUT /identity/v3/projects/2ffa0c4279a84e0085384a8f72c1fdc8/users/cfa8d8d688bd4560a66df89bd5ec20dd/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:28:45.564188 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:28:45.574380 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c71efdd-a708-493f-abef-7f9cdd17d422 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:45.574642 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1079/2189] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:28:45 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 06:28:45.575455 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c71efdd-a708-493f-abef-7f9cdd17d422 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:45.575934 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c71efdd-a708-493f-abef-7f9cdd17d422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:45.650912 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-4c71efdd-a708-493f-abef-7f9cdd17d422 None None] MFA Rules not processed for user `cfa8d8d688bd4560a66df89bd5ec20dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:28:45.653860 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4c71efdd-a708-493f-abef-7f9cdd17d422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:45.816814 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1111/2190] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:28:45 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:28:45.838717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60533109-4f9c-4fbd-bbec-68507f747948 service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:45.850146 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60533109-4f9c-4fbd-bbec-68507f747948 service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:45.919919 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073aa9e-5fa5-4cca-ba3f-855e3c605d73 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:45.920490 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073aa9e-5fa5-4cca-ba3f-855e3c605d73 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:45.920604 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073aa9e-5fa5-4cca-ba3f-855e3c605d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:45.930352 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d6227-0d41-42dd-9972-c237b13a0933 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:45.931035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d6227-0d41-42dd-9972-c237b13a0933 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:45.931330 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d6227-0d41-42dd-9972-c237b13a0933 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:45.931592 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d6227-0d41-42dd-9972-c237b13a0933 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:45.942904 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d6227-0d41-42dd-9972-c237b13a0933 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:45.944356 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d6227-0d41-42dd-9972-c237b13a0933 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:45.956985 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-5073aa9e-5fa5-4cca-ba3f-855e3c605d73 None None] MFA Rules not processed for user `936ea0959e924aac95cf474687e35c78`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:28:45.958600 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-5073aa9e-5fa5-4cca-ba3f-855e3c605d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:45.959776 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1080/2191] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:28:45 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:28:46.055871 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1112/2192] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:28:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:28:46.320458 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d6227-0d41-42dd-9972-c237b13a0933 service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:46.335286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d6227-0d41-42dd-9972-c237b13a0933 service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:46.339659 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a7474-4de1-4dfe-bf7d-772400d56f20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:46.340642 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a7474-4de1-4dfe-bf7d-772400d56f20 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:46.341083 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a7474-4de1-4dfe-bf7d-772400d56f20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:46.341510 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a7474-4de1-4dfe-bf7d-772400d56f20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:46.350375 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:28:46.451222 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a7474-4de1-4dfe-bf7d-772400d56f20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:46.454376 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a7474-4de1-4dfe-bf7d-772400d56f20 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:46.467560 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1081/2193] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:28:46 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:28:46.935431 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:28:47.186169 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:28:47.620958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:28:47.843701 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:28:57.957736 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=23 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:28:58.112694 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=20 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:28:59.442984 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073aa9e-5fa5-4cca-ba3f-855e3c605d73 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:59.526446 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073aa9e-5fa5-4cca-ba3f-855e3c605d73 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:28:59.531662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81c0a4-b8ea-4aeb-8e5a-1061d08c3ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:28:59.532841 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81c0a4-b8ea-4aeb-8e5a-1061d08c3ecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:59.533125 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81c0a4-b8ea-4aeb-8e5a-1061d08c3ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:59.533460 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81c0a4-b8ea-4aeb-8e5a-1061d08c3ecc admin admin] PATH_INFO: `/v3/users/400a779e2f8f4c14b873fa49fd3a8cfe` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:59.534753 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d81c0a4-b8ea-4aeb-8e5a-1061d08c3ecc admin admin] RBAC: Authorizing `identity:delete_user(user_id=400a779e2f8f4c14b873fa49fd3a8cfe)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:28:59.536765 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d81c0a4-b8ea-4aeb-8e5a-1061d08c3ecc admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:28:59.744422 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1d81c0a4-b8ea-4aeb-8e5a-1061d08c3ecc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '400a779e2f8f4c14b873fa49fd3a8cfe'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:59.751779 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1d81c0a4-b8ea-4aeb-8e5a-1061d08c3ecc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '400a779e2f8f4c14b873fa49fd3a8cfe'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:59.752543 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1d81c0a4-b8ea-4aeb-8e5a-1061d08c3ecc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '400a779e2f8f4c14b873fa49fd3a8cfe'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:59.753180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1d81c0a4-b8ea-4aeb-8e5a-1061d08c3ecc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '400a779e2f8f4c14b873fa49fd3a8cfe'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:59.766307 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1d81c0a4-b8ea-4aeb-8e5a-1061d08c3ecc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '400a779e2f8f4c14b873fa49fd3a8cfe'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:28:59.832780 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1113/2194] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:28:59 2022] DELETE /identity/v3/users/400a779e2f8f4c14b873fa49fd3a8cfe => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:28:59.837425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c5b16-d319-44e4-9f10-c13ef7c899a3 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:28:59.837942 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c5b16-d319-44e4-9f10-c13ef7c899a3 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:28:59.838064 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c5b16-d319-44e4-9f10-c13ef7c899a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:28:59.856798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-3f9c5b16-d319-44e4-9f10-c13ef7c899a3 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:28:59.858021 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-3f9c5b16-d319-44e4-9f10-c13ef7c899a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:28:59.939589 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1082/2195] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:28:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:28:59.949735 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81c0a4-b8ea-4aeb-8e5a-1061d08c3ecc admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:28:59.951270 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-1d81c0a4-b8ea-4aeb-8e5a-1061d08c3ecc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:00.029337 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81c0a4-b8ea-4aeb-8e5a-1061d08c3ecc admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:00.035033 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d96b53-4641-45db-8072-f6a9cbbae78b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:00.035939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d96b53-4641-45db-8072-f6a9cbbae78b service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:00.036471 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d96b53-4641-45db-8072-f6a9cbbae78b service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:00.036909 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d96b53-4641-45db-8072-f6a9cbbae78b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:00.052651 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d96b53-4641-45db-8072-f6a9cbbae78b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:00.058007 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d96b53-4641-45db-8072-f6a9cbbae78b service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:00.073604 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1114/2196] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:28:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 06:29:00.674345 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c5b16-d319-44e4-9f10-c13ef7c899a3 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:00.676395 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-3f9c5b16-d319-44e4-9f10-c13ef7c899a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:00.692809 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c5b16-d319-44e4-9f10-c13ef7c899a3 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:00.698207 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed40b2a-6e06-420f-b0b0-65e7bb4a1964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:00.699104 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed40b2a-6e06-420f-b0b0-65e7bb4a1964 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:00.699394 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed40b2a-6e06-420f-b0b0-65e7bb4a1964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:00.699731 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed40b2a-6e06-420f-b0b0-65e7bb4a1964 admin admin] PATH_INFO: `/v3/projects/367b98c53f844498bebdc4be8392d92a` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:00.701128 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed40b2a-6e06-420f-b0b0-65e7bb4a1964 admin admin] RBAC: Authorizing `identity:delete_project(project_id=367b98c53f844498bebdc4be8392d92a)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:00.704130 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed40b2a-6e06-420f-b0b0-65e7bb4a1964 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:00.810414 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-5ed40b2a-6e06-420f-b0b0-65e7bb4a1964 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '367b98c53f844498bebdc4be8392d92a', 'request_id': 'req-5ed40b2a-6e06-420f-b0b0-65e7bb4a1964'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:00.816035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-5ed40b2a-6e06-420f-b0b0-65e7bb4a1964 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '367b98c53f844498bebdc4be8392d92a', 'request_id': 'req-5ed40b2a-6e06-420f-b0b0-65e7bb4a1964'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:00.888693 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-5ed40b2a-6e06-420f-b0b0-65e7bb4a1964 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '367b98c53f844498bebdc4be8392d92a', 'request_id': 'req-5ed40b2a-6e06-420f-b0b0-65e7bb4a1964'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:00.901168 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-5ed40b2a-6e06-420f-b0b0-65e7bb4a1964 admin admin] The token cache is being invalidate because project 367b98c53f844498bebdc4be8392d92a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:29:00.901422 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-5ed40b2a-6e06-420f-b0b0-65e7bb4a1964 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:00.902074 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-5ed40b2a-6e06-420f-b0b0-65e7bb4a1964 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:00.903614 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1083/2197] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:29:00 2022] DELETE /identity/v3/projects/367b98c53f844498bebdc4be8392d92a => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:29:00.908280 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d96b53-4641-45db-8072-f6a9cbbae78b service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:00.918026 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d96b53-4641-45db-8072-f6a9cbbae78b service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:00.936109 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:00.976406 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b6440a-99f3-4e62-b6e9-9aa5550cc9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:00.977685 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6440a-99f3-4e62-b6e9-9aa5550cc9df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:00.977980 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6440a-99f3-4e62-b6e9-9aa5550cc9df admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:00.978856 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6440a-99f3-4e62-b6e9-9aa5550cc9df admin admin] PATH_INFO: `/v3/users/3ab4966dba1b4fc6a95129d5ebb033fb` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:00.980250 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b6440a-99f3-4e62-b6e9-9aa5550cc9df admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ab4966dba1b4fc6a95129d5ebb033fb)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:00.982312 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b6440a-99f3-4e62-b6e9-9aa5550cc9df admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:01.194765 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-74b6440a-99f3-4e62-b6e9-9aa5550cc9df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ab4966dba1b4fc6a95129d5ebb033fb'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:01.202483 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-74b6440a-99f3-4e62-b6e9-9aa5550cc9df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ab4966dba1b4fc6a95129d5ebb033fb'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:01.203805 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-74b6440a-99f3-4e62-b6e9-9aa5550cc9df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ab4966dba1b4fc6a95129d5ebb033fb'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:01.204586 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-74b6440a-99f3-4e62-b6e9-9aa5550cc9df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ab4966dba1b4fc6a95129d5ebb033fb'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:01.221350 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-74b6440a-99f3-4e62-b6e9-9aa5550cc9df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ab4966dba1b4fc6a95129d5ebb033fb'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:01.295157 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1115/2198] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:29:00 2022] DELETE /identity/v3/users/3ab4966dba1b4fc6a95129d5ebb033fb => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:29:01.460776 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed40b2a-6e06-420f-b0b0-65e7bb4a1964 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:01.462291 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-5ed40b2a-6e06-420f-b0b0-65e7bb4a1964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:01.476518 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed40b2a-6e06-420f-b0b0-65e7bb4a1964 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:01.561979 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084569ed-503c-40f4-923f-05c397ef641a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:01.567157 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084569ed-503c-40f4-923f-05c397ef641a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:01.567157 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084569ed-503c-40f4-923f-05c397ef641a admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:01.567157 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084569ed-503c-40f4-923f-05c397ef641a admin admin] PATH_INFO: `/v3/users/8a19ecad5df944419a5aa3652921f2a9` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:01.567157 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084569ed-503c-40f4-923f-05c397ef641a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a19ecad5df944419a5aa3652921f2a9)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:01.568909 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084569ed-503c-40f4-923f-05c397ef641a admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:01.667320 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:01.776207 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-084569ed-503c-40f4-923f-05c397ef641a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a19ecad5df944419a5aa3652921f2a9'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:01.789986 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-084569ed-503c-40f4-923f-05c397ef641a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a19ecad5df944419a5aa3652921f2a9'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:01.809092 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-084569ed-503c-40f4-923f-05c397ef641a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a19ecad5df944419a5aa3652921f2a9'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:01.809881 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-084569ed-503c-40f4-923f-05c397ef641a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a19ecad5df944419a5aa3652921f2a9'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:01.810668 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-084569ed-503c-40f4-923f-05c397ef641a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a19ecad5df944419a5aa3652921f2a9'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:01.836705 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b6440a-99f3-4e62-b6e9-9aa5550cc9df admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:01.838155 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-74b6440a-99f3-4e62-b6e9-9aa5550cc9df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:01.852023 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b6440a-99f3-4e62-b6e9-9aa5550cc9df admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:01.867410 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd8e64-ecbe-47a5-89ae-3b9ac4ca03dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:01.868036 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd8e64-ecbe-47a5-89ae-3b9ac4ca03dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:01.868330 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd8e64-ecbe-47a5-89ae-3b9ac4ca03dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:01.868657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd8e64-ecbe-47a5-89ae-3b9ac4ca03dd admin admin] PATH_INFO: `/v3/projects/c00d53c8ca3741c7b67d2234c24915e9` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:01.870674 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfd8e64-ecbe-47a5-89ae-3b9ac4ca03dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c00d53c8ca3741c7b67d2234c24915e9)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:01.872666 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfd8e64-ecbe-47a5-89ae-3b9ac4ca03dd admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:01.922272 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1084/2199] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:29:01 2022] DELETE /identity/v3/users/8a19ecad5df944419a5aa3652921f2a9 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:29:01.922272 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9070858c-2947-4f33-a423-af856e343d4c None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:01.922272 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9070858c-2947-4f33-a423-af856e343d4c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:01.922927 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9070858c-2947-4f33-a423-af856e343d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:01.940691 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-9070858c-2947-4f33-a423-af856e343d4c None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:01.941914 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-9070858c-2947-4f33-a423-af856e343d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:02.001524 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-2cfd8e64-ecbe-47a5-89ae-3b9ac4ca03dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c00d53c8ca3741c7b67d2234c24915e9', 'request_id': 'req-2cfd8e64-ecbe-47a5-89ae-3b9ac4ca03dd'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:02.007962 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-2cfd8e64-ecbe-47a5-89ae-3b9ac4ca03dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c00d53c8ca3741c7b67d2234c24915e9', 'request_id': 'req-2cfd8e64-ecbe-47a5-89ae-3b9ac4ca03dd'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:02.014105 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1085/2200] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:29:02.023139 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9070858c-2947-4f33-a423-af856e343d4c None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:02.024534 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-9070858c-2947-4f33-a423-af856e343d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:02.085707 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9070858c-2947-4f33-a423-af856e343d4c None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:02.089604 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-2cfd8e64-ecbe-47a5-89ae-3b9ac4ca03dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c00d53c8ca3741c7b67d2234c24915e9', 'request_id': 'req-2cfd8e64-ecbe-47a5-89ae-3b9ac4ca03dd'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:02.107858 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-2cfd8e64-ecbe-47a5-89ae-3b9ac4ca03dd admin admin] The token cache is being invalidate because project c00d53c8ca3741c7b67d2234c24915e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:29:02.110268 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-2cfd8e64-ecbe-47a5-89ae-3b9ac4ca03dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:02.111436 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-2cfd8e64-ecbe-47a5-89ae-3b9ac4ca03dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:02.113262 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1116/2201] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:29:01 2022] DELETE /identity/v3/projects/c00d53c8ca3741c7b67d2234c24915e9 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:29:02.119207 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd217f-aaf1-4f81-b76d-764c8d0992ee None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:02.119421 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd217f-aaf1-4f81-b76d-764c8d0992ee None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:02.119708 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd217f-aaf1-4f81-b76d-764c8d0992ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:02.147199 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f9b32-70b8-4588-bccc-e010e694a800 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:02.147628 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-e7fd217f-aaf1-4f81-b76d-764c8d0992ee None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:02.148595 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f9b32-70b8-4588-bccc-e010e694a800 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:02.148595 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f9b32-70b8-4588-bccc-e010e694a800 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:02.148900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f9b32-70b8-4588-bccc-e010e694a800 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:02.149005 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e7fd217f-aaf1-4f81-b76d-764c8d0992ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:02.150955 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b88f9b32-70b8-4588-bccc-e010e694a800 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:02.172197 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:02.220373 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1117/2202] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:29:02.228156 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba8985-ac42-4507-95ac-8dda3a87e1d0 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:02.228500 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba8985-ac42-4507-95ac-8dda3a87e1d0 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:02.228793 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba8985-ac42-4507-95ac-8dda3a87e1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:02.232893 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88f9b32-70b8-4588-bccc-e010e694a800 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:02.235457 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88f9b32-70b8-4588-bccc-e010e694a800 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:02.252714 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-8dba8985-ac42-4507-95ac-8dda3a87e1d0 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:02.253071 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1086/2203] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:29:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 06:29:02.257847 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-8dba8985-ac42-4507-95ac-8dda3a87e1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:02.269602 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1118/2204] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:29:02.274078 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f9b32-70b8-4588-bccc-e010e694a800 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:02.286479 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f9b32-70b8-4588-bccc-e010e694a800 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:02.298501 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1728d134-267e-40ca-8546-3057b08045dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:02.298501 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728d134-267e-40ca-8546-3057b08045dd admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:02.298501 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728d134-267e-40ca-8546-3057b08045dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:02.298501 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728d134-267e-40ca-8546-3057b08045dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:02.298846 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1728d134-267e-40ca-8546-3057b08045dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:29:02.298846 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1728d134-267e-40ca-8546-3057b08045dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:02.299984 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1728d134-267e-40ca-8546-3057b08045dd admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:02.333438 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1087/2205] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:29:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:29:02.334942 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba8985-ac42-4507-95ac-8dda3a87e1d0 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:02.349191 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba8985-ac42-4507-95ac-8dda3a87e1d0 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:02.357227 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f34c76-e9b9-42c1-9a31-f3a254efdd0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:02.359130 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f34c76-e9b9-42c1-9a31-f3a254efdd0c admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:02.359874 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f34c76-e9b9-42c1-9a31-f3a254efdd0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:02.360488 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f34c76-e9b9-42c1-9a31-f3a254efdd0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:02.361464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f34c76-e9b9-42c1-9a31-f3a254efdd0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:02.363787 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f34c76-e9b9-42c1-9a31-f3a254efdd0c admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:02.499396 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1119/2206] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:29:02 2022] POST /identity/v3/projects => generated 381 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:29:02.508170 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1728d134-267e-40ca-8546-3057b08045dd admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:02.546907 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1728d134-267e-40ca-8546-3057b08045dd admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:02.638318 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536f339-7fdb-48fa-b4f0-9f69befdb6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:02.639000 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536f339-7fdb-48fa-b4f0-9f69befdb6d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:02.639286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536f339-7fdb-48fa-b4f0-9f69befdb6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:02.639570 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536f339-7fdb-48fa-b4f0-9f69befdb6d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:02.640994 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536f339-7fdb-48fa-b4f0-9f69befdb6d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:02.643201 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536f339-7fdb-48fa-b4f0-9f69befdb6d0 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:02.741390 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1088/2207] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:29:02 2022] POST /identity/v3/users => generated 291 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:29:02.745906 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f34c76-e9b9-42c1-9a31-f3a254efdd0c admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:02.757172 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f34c76-e9b9-42c1-9a31-f3a254efdd0c admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:02.761372 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c215f81-60c4-420f-8d70-5b819530081d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:02.762049 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c215f81-60c4-420f-8d70-5b819530081d admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:02.762309 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c215f81-60c4-420f-8d70-5b819530081d admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:02.762586 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c215f81-60c4-420f-8d70-5b819530081d admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:02.763308 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c215f81-60c4-420f-8d70-5b819530081d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:02.764950 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c215f81-60c4-420f-8d70-5b819530081d admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:02.776160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1120/2208] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:29:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 06:29:02.780228 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536f339-7fdb-48fa-b4f0-9f69befdb6d0 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:02.790366 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536f339-7fdb-48fa-b4f0-9f69befdb6d0 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:02.793146 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:02.800185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c215f81-60c4-420f-8d70-5b819530081d admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:02.802148 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3b7b5-1103-4d2e-94e4-08cedbc161c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:02.802304 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-5c215f81-60c4-420f-8d70-5b819530081d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:02.802847 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3b7b5-1103-4d2e-94e4-08cedbc161c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:02.803137 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3b7b5-1103-4d2e-94e4-08cedbc161c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:02.803419 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3b7b5-1103-4d2e-94e4-08cedbc161c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:02.804268 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be3b7b5-1103-4d2e-94e4-08cedbc161c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:02.806080 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be3b7b5-1103-4d2e-94e4-08cedbc161c9 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:02.816733 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c215f81-60c4-420f-8d70-5b819530081d admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:02.822092 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac64a2-22e1-4812-a10a-7364c54b8cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:02.822966 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac64a2-22e1-4812-a10a-7364c54b8cb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:02.826623 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac64a2-22e1-4812-a10a-7364c54b8cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:02.826740 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1089/2209] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:29:02 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:29:02.826835 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac64a2-22e1-4812-a10a-7364c54b8cb4 admin admin] PATH_INFO: `/v3/projects/69f190aaf4b84ebeb71646e88ba352d6` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:02.826835 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac64a2-22e1-4812-a10a-7364c54b8cb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69f190aaf4b84ebeb71646e88ba352d6)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:02.826835 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac64a2-22e1-4812-a10a-7364c54b8cb4 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:02.828326 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3b7b5-1103-4d2e-94e4-08cedbc161c9 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:02.838988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3b7b5-1103-4d2e-94e4-08cedbc161c9 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:02.844173 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c053da6-6857-40eb-9a31-4f3475b52e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:02.845349 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c053da6-6857-40eb-9a31-4f3475b52e0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:02.845648 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c053da6-6857-40eb-9a31-4f3475b52e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:02.845928 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c053da6-6857-40eb-9a31-4f3475b52e0f admin admin] PATH_INFO: `/v3/projects/3515fcef23ba4c7f9224404a96dc1b7c/users/87ce6e5f3976486fad9a6991e93590da/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:02.863259 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c053da6-6857-40eb-9a31-4f3475b52e0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3515fcef23ba4c7f9224404a96dc1b7c, user_id=87ce6e5f3976486fad9a6991e93590da, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:02.866090 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c053da6-6857-40eb-9a31-4f3475b52e0f admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:02.954577 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1090/2210] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:29:02 2022] PUT /identity/v3/projects/3515fcef23ba4c7f9224404a96dc1b7c/users/87ce6e5f3976486fad9a6991e93590da/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:29:02.965978 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b52a00-54ba-406d-aa2b-9807f772f68d None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:02.966279 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b52a00-54ba-406d-aa2b-9807f772f68d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:02.966566 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b52a00-54ba-406d-aa2b-9807f772f68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:02.969599 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-dcac64a2-22e1-4812-a10a-7364c54b8cb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69f190aaf4b84ebeb71646e88ba352d6', 'request_id': 'req-dcac64a2-22e1-4812-a10a-7364c54b8cb4'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:02.978474 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-dcac64a2-22e1-4812-a10a-7364c54b8cb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69f190aaf4b84ebeb71646e88ba352d6', 'request_id': 'req-dcac64a2-22e1-4812-a10a-7364c54b8cb4'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:03.002597 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-96b52a00-54ba-406d-aa2b-9807f772f68d None None] MFA Rules not processed for user `87ce6e5f3976486fad9a6991e93590da`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:03.003986 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-96b52a00-54ba-406d-aa2b-9807f772f68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:03.049244 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-dcac64a2-22e1-4812-a10a-7364c54b8cb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69f190aaf4b84ebeb71646e88ba352d6', 'request_id': 'req-dcac64a2-22e1-4812-a10a-7364c54b8cb4'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:03.063449 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-dcac64a2-22e1-4812-a10a-7364c54b8cb4 admin admin] The token cache is being invalidate because project 69f190aaf4b84ebeb71646e88ba352d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:29:03.063956 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-dcac64a2-22e1-4812-a10a-7364c54b8cb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:03.064845 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-dcac64a2-22e1-4812-a10a-7364c54b8cb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:03.067228 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1121/2211] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:29:02 2022] DELETE /identity/v3/projects/69f190aaf4b84ebeb71646e88ba352d6 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:29:03.072586 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fb834-d359-40dc-a329-1e9b2f1e3e38 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:03.072880 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fb834-d359-40dc-a329-1e9b2f1e3e38 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:03.073168 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fb834-d359-40dc-a329-1e9b2f1e3e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:03.097098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1091/2212] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:29:03.098965 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-721fb834-d359-40dc-a329-1e9b2f1e3e38 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:03.100212 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-721fb834-d359-40dc-a329-1e9b2f1e3e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:03.103216 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b52a00-54ba-406d-aa2b-9807f772f68d None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:03.105944 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-96b52a00-54ba-406d-aa2b-9807f772f68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:03.246339 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1122/2213] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:29:03.247346 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b52a00-54ba-406d-aa2b-9807f772f68d None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:03.247346 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b406278-a5e2-4118-b2df-c6dc130b7e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:03.247346 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b406278-a5e2-4118-b2df-c6dc130b7e53 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:03.247346 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b406278-a5e2-4118-b2df-c6dc130b7e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:03.248922 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b406278-a5e2-4118-b2df-c6dc130b7e53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:03.248922 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b406278-a5e2-4118-b2df-c6dc130b7e53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:03.248922 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b406278-a5e2-4118-b2df-c6dc130b7e53 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:03.250867 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966e66b-f07c-425b-852a-99f8f7364fde None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:03.250867 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966e66b-f07c-425b-852a-99f8f7364fde None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:03.250867 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966e66b-f07c-425b-852a-99f8f7364fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:03.271747 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-0966e66b-f07c-425b-852a-99f8f7364fde None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:03.274736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-0966e66b-f07c-425b-852a-99f8f7364fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:03.286100 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1123/2214] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:29:03.287594 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0966e66b-f07c-425b-852a-99f8f7364fde None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:03.298223 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0966e66b-f07c-425b-852a-99f8f7364fde None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:03.303352 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d7649d-bceb-4231-b4c7-52766845a3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:03.304043 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7649d-bceb-4231-b4c7-52766845a3e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:03.304327 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7649d-bceb-4231-b4c7-52766845a3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:03.304639 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7649d-bceb-4231-b4c7-52766845a3e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:03.305393 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d7649d-bceb-4231-b4c7-52766845a3e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:29:03.305780 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d7649d-bceb-4231-b4c7-52766845a3e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:03.307376 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d7649d-bceb-4231-b4c7-52766845a3e0 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:03.322438 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1124/2215] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:29:03.329613 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d7649d-bceb-4231-b4c7-52766845a3e0 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:03.340386 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d7649d-bceb-4231-b4c7-52766845a3e0 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:03.346027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a672b0a-d1d4-4b5c-bf83-998667f7eaa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:03.346817 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a672b0a-d1d4-4b5c-bf83-998667f7eaa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:03.347131 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a672b0a-d1d4-4b5c-bf83-998667f7eaa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:03.347415 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a672b0a-d1d4-4b5c-bf83-998667f7eaa5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:03.348133 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a672b0a-d1d4-4b5c-bf83-998667f7eaa5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:03.350161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a672b0a-d1d4-4b5c-bf83-998667f7eaa5 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:03.664495 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:03.677341 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1092/2216] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:29:03 2022] POST /identity/v3/projects => generated 385 bytes in 575 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:29:03.682044 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b406278-a5e2-4118-b2df-c6dc130b7e53 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:03.774094 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1125/2217] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:29:03 2022] POST /identity/v3/projects => generated 373 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:29:03.778966 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a672b0a-d1d4-4b5c-bf83-998667f7eaa5 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:03.812720 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b406278-a5e2-4118-b2df-c6dc130b7e53 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:03.882416 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a672b0a-d1d4-4b5c-bf83-998667f7eaa5 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:03.888447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06093fdf-0397-4600-b4db-1ecb12462810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:03.894635 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06093fdf-0397-4600-b4db-1ecb12462810 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:03.894635 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06093fdf-0397-4600-b4db-1ecb12462810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:03.894635 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06093fdf-0397-4600-b4db-1ecb12462810 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:03.894635 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06093fdf-0397-4600-b4db-1ecb12462810 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:03.894635 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06093fdf-0397-4600-b4db-1ecb12462810 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:03.905041 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae36d64-e4fa-41b3-a2a7-748c6126a225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:03.908482 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae36d64-e4fa-41b3-a2a7-748c6126a225 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:03.909048 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae36d64-e4fa-41b3-a2a7-748c6126a225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:03.909504 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae36d64-e4fa-41b3-a2a7-748c6126a225 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:03.912473 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae36d64-e4fa-41b3-a2a7-748c6126a225 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:03.914337 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae36d64-e4fa-41b3-a2a7-748c6126a225 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:03.992685 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1126/2218] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:29:03 2022] POST /identity/v3/users => generated 287 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:29:03.992685 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06093fdf-0397-4600-b4db-1ecb12462810 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:04.002804 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06093fdf-0397-4600-b4db-1ecb12462810 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:04.007986 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5172c20c-e436-4fda-a217-cbf14dd2de9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:04.008714 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172c20c-e436-4fda-a217-cbf14dd2de9e admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:04.009212 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172c20c-e436-4fda-a217-cbf14dd2de9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:04.009330 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172c20c-e436-4fda-a217-cbf14dd2de9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:04.010242 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5172c20c-e436-4fda-a217-cbf14dd2de9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:04.011679 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5172c20c-e436-4fda-a217-cbf14dd2de9e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:04.016753 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1093/2219] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:29:03 2022] POST /identity/v3/users => generated 293 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:29:04.025651 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1127/2220] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:29:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 06:29:04.025807 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae36d64-e4fa-41b3-a2a7-748c6126a225 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:04.027132 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5172c20c-e436-4fda-a217-cbf14dd2de9e admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:04.035716 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae36d64-e4fa-41b3-a2a7-748c6126a225 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:04.038360 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5172c20c-e436-4fda-a217-cbf14dd2de9e admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:04.040617 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2b81cc-3379-4a00-affc-accc9eb79070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:04.041290 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b81cc-3379-4a00-affc-accc9eb79070 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:04.041569 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b81cc-3379-4a00-affc-accc9eb79070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:04.041880 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b81cc-3379-4a00-affc-accc9eb79070 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:04.042784 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2b81cc-3379-4a00-affc-accc9eb79070 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:04.044043 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bcf358-bfc8-4d08-aea7-daa781b8e163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:04.044563 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2b81cc-3379-4a00-affc-accc9eb79070 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:04.044735 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bcf358-bfc8-4d08-aea7-daa781b8e163 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:04.045081 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bcf358-bfc8-4d08-aea7-daa781b8e163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:04.045362 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bcf358-bfc8-4d08-aea7-daa781b8e163 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:04.046365 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bcf358-bfc8-4d08-aea7-daa781b8e163 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:04.048786 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bcf358-bfc8-4d08-aea7-daa781b8e163 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:04.056839 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1094/2221] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 06:29:04.060678 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1128/2222] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 06:29:04.061316 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2b81cc-3379-4a00-affc-accc9eb79070 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:04.065313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bcf358-bfc8-4d08-aea7-daa781b8e163 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:04.071956 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2b81cc-3379-4a00-affc-accc9eb79070 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:04.076558 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bcf358-bfc8-4d08-aea7-daa781b8e163 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:04.083881 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb3663e-d059-49de-8f06-f2368515938a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:04.083881 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3663e-d059-49de-8f06-f2368515938a admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:04.083881 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3663e-d059-49de-8f06-f2368515938a admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:04.083881 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3663e-d059-49de-8f06-f2368515938a admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:04.084284 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb3663e-d059-49de-8f06-f2368515938a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:04.084284 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb3663e-d059-49de-8f06-f2368515938a admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:04.084467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88453eae-aee4-461b-ae94-f5a961a79d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:04.084467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88453eae-aee4-461b-ae94-f5a961a79d3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:04.084467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88453eae-aee4-461b-ae94-f5a961a79d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:04.084467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88453eae-aee4-461b-ae94-f5a961a79d3b admin admin] PATH_INFO: `/v3/projects/8031b2e212154353a53ea00aeb6b6eea/users/5ddb5958dbd94963b7a7e8baecfcb020/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:04.092440 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1095/2223] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 06:29:04.097823 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb3663e-d059-49de-8f06-f2368515938a admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:04.101730 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88453eae-aee4-461b-ae94-f5a961a79d3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8031b2e212154353a53ea00aeb6b6eea, user_id=5ddb5958dbd94963b7a7e8baecfcb020, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:04.104521 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88453eae-aee4-461b-ae94-f5a961a79d3b admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:04.110381 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb3663e-d059-49de-8f06-f2368515938a admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:04.114205 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226cfee2-3b36-4144-ad1a-83a0312b01cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:04.115102 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cfee2-3b36-4144-ad1a-83a0312b01cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:04.115385 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cfee2-3b36-4144-ad1a-83a0312b01cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:04.115709 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cfee2-3b36-4144-ad1a-83a0312b01cb admin admin] PATH_INFO: `/v3/projects/ae587556ebd1482aa353f60d58bcd89c/users/40468f6c404f40708a0a48863bdfc316/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:04.131368 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226cfee2-3b36-4144-ad1a-83a0312b01cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae587556ebd1482aa353f60d58bcd89c, user_id=40468f6c404f40708a0a48863bdfc316, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:04.134159 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226cfee2-3b36-4144-ad1a-83a0312b01cb admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:04.137925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:04.240073 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1096/2224] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:29:04 2022] PUT /identity/v3/projects/ae587556ebd1482aa353f60d58bcd89c/users/40468f6c404f40708a0a48863bdfc316/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:29:04.240548 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1129/2225] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:29:04 2022] PUT /identity/v3/projects/8031b2e212154353a53ea00aeb6b6eea/users/5ddb5958dbd94963b7a7e8baecfcb020/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:29:04.258697 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb549b-4e53-414d-a3e2-94d04cf66acc None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:04.259209 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb549b-4e53-414d-a3e2-94d04cf66acc None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:04.259652 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb549b-4e53-414d-a3e2-94d04cf66acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:04.295317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-c2bb549b-4e53-414d-a3e2-94d04cf66acc None None] MFA Rules not processed for user `5ddb5958dbd94963b7a7e8baecfcb020`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:04.296798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-c2bb549b-4e53-414d-a3e2-94d04cf66acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:04.314651 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc5f08-948a-4296-a34c-b30046cdc852 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:04.315197 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc5f08-948a-4296-a34c-b30046cdc852 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:04.316767 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc5f08-948a-4296-a34c-b30046cdc852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:04.361931 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-f3dc5f08-948a-4296-a34c-b30046cdc852 None None] MFA Rules not processed for user `40468f6c404f40708a0a48863bdfc316`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:04.363661 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-f3dc5f08-948a-4296-a34c-b30046cdc852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:04.380087 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1097/2226] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:29:04.394072 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb549b-4e53-414d-a3e2-94d04cf66acc None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:04.396232 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-c2bb549b-4e53-414d-a3e2-94d04cf66acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:04.443966 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1130/2227] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:29:04.457470 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dc5f08-948a-4296-a34c-b30046cdc852 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:04.465299 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb549b-4e53-414d-a3e2-94d04cf66acc None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:04.470919 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603934de-a971-42c2-9f24-40d21816c4d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:04.471993 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603934de-a971-42c2-9f24-40d21816c4d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:04.472289 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603934de-a971-42c2-9f24-40d21816c4d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:04.472609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603934de-a971-42c2-9f24-40d21816c4d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:04.483178 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603934de-a971-42c2-9f24-40d21816c4d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:04.484995 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603934de-a971-42c2-9f24-40d21816c4d3 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:04.508068 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1098/2228] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:29:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 06:29:04.538272 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603934de-a971-42c2-9f24-40d21816c4d3 service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:04.541940 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dc5f08-948a-4296-a34c-b30046cdc852 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:04.547750 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adca63b-2209-4195-a1b1-67f1cf9d4317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:04.548366 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adca63b-2209-4195-a1b1-67f1cf9d4317 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:04.548712 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adca63b-2209-4195-a1b1-67f1cf9d4317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:04.548977 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adca63b-2209-4195-a1b1-67f1cf9d4317 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:04.549665 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adca63b-2209-4195-a1b1-67f1cf9d4317 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:04.551641 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adca63b-2209-4195-a1b1-67f1cf9d4317 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:04.552028 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603934de-a971-42c2-9f24-40d21816c4d3 service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:04.559711 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c83883d-deca-447d-8338-ed8b0802f486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:04.562793 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c83883d-deca-447d-8338-ed8b0802f486 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:04.562793 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c83883d-deca-447d-8338-ed8b0802f486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:04.562793 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c83883d-deca-447d-8338-ed8b0802f486 admin admin] PATH_INFO: `/v3/users/5ddb5958dbd94963b7a7e8baecfcb020` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:04.564806 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c83883d-deca-447d-8338-ed8b0802f486 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ddb5958dbd94963b7a7e8baecfcb020)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:04.567199 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c83883d-deca-447d-8338-ed8b0802f486 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:04.575269 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:04.640625 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1131/2229] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:29:04 2022] POST /identity/v3/projects => generated 383 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:29:04.642119 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adca63b-2209-4195-a1b1-67f1cf9d4317 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:04.690202 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:04.711693 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adca63b-2209-4195-a1b1-67f1cf9d4317 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:04.716606 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ccbcf9-86ea-46f6-9c72-cc21777f60fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:04.717237 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ccbcf9-86ea-46f6-9c72-cc21777f60fd admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:04.717523 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ccbcf9-86ea-46f6-9c72-cc21777f60fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:04.717801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ccbcf9-86ea-46f6-9c72-cc21777f60fd admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:04.718574 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ccbcf9-86ea-46f6-9c72-cc21777f60fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:04.720753 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ccbcf9-86ea-46f6-9c72-cc21777f60fd admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:04.767612 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-2c83883d-deca-447d-8338-ed8b0802f486 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ddb5958dbd94963b7a7e8baecfcb020'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:04.775247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-2c83883d-deca-447d-8338-ed8b0802f486 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ddb5958dbd94963b7a7e8baecfcb020'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:04.787572 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-2c83883d-deca-447d-8338-ed8b0802f486 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ddb5958dbd94963b7a7e8baecfcb020'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:04.789073 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-2c83883d-deca-447d-8338-ed8b0802f486 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ddb5958dbd94963b7a7e8baecfcb020'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:04.789073 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-2c83883d-deca-447d-8338-ed8b0802f486 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ddb5958dbd94963b7a7e8baecfcb020'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:04.844952 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1132/2230] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:29:04 2022] POST /identity/v3/users => generated 298 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:29:04.849152 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ccbcf9-86ea-46f6-9c72-cc21777f60fd admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:04.851199 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e4ccbcf9-86ea-46f6-9c72-cc21777f60fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:04.865120 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ccbcf9-86ea-46f6-9c72-cc21777f60fd admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:04.870869 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f494ede-9861-429e-9837-996a96330d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:04.871645 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f494ede-9861-429e-9837-996a96330d6d admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:04.872005 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f494ede-9861-429e-9837-996a96330d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:04.872385 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f494ede-9861-429e-9837-996a96330d6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:04.873251 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f494ede-9861-429e-9837-996a96330d6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:04.875765 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f494ede-9861-429e-9837-996a96330d6d admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:04.877241 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1099/2231] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:29:04 2022] DELETE /identity/v3/users/5ddb5958dbd94963b7a7e8baecfcb020 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:29:04.886878 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a558d-513b-4bca-964d-f9ad76bf5fde None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:04.886878 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a558d-513b-4bca-964d-f9ad76bf5fde None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:04.886878 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a558d-513b-4bca-964d-f9ad76bf5fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:04.895880 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1133/2232] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:29:04.895880 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f494ede-9861-429e-9837-996a96330d6d admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:04.907376 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f494ede-9861-429e-9837-996a96330d6d admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:04.923406 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-292a558d-513b-4bca-964d-f9ad76bf5fde None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:04.924706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-292a558d-513b-4bca-964d-f9ad76bf5fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:04.981714 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364c458-c70c-4bc9-a6c9-86297efb0d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:04.982624 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364c458-c70c-4bc9-a6c9-86297efb0d16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:04.982912 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364c458-c70c-4bc9-a6c9-86297efb0d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:04.983194 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364c458-c70c-4bc9-a6c9-86297efb0d16 admin admin] PATH_INFO: `/v3/projects/c50588dbcfd54bb0b28556064c9b2d50/users/73432fdcfb0446dcb6e43e2e56a14ca2/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:04.993172 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1100/2233] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:29:05.006231 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a558d-513b-4bca-964d-f9ad76bf5fde None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:05.007972 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1364c458-c70c-4bc9-a6c9-86297efb0d16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c50588dbcfd54bb0b28556064c9b2d50, user_id=73432fdcfb0446dcb6e43e2e56a14ca2, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:05.008558 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-292a558d-513b-4bca-964d-f9ad76bf5fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:05.010908 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1364c458-c70c-4bc9-a6c9-86297efb0d16 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:05.016596 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:05.079380 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a558d-513b-4bca-964d-f9ad76bf5fde None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:05.083302 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5eadfc-882b-4645-818e-1f69ce9e68f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:05.083973 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5eadfc-882b-4645-818e-1f69ce9e68f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:05.084258 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5eadfc-882b-4645-818e-1f69ce9e68f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:05.084581 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5eadfc-882b-4645-818e-1f69ce9e68f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:05.104801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5eadfc-882b-4645-818e-1f69ce9e68f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:05.104801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5eadfc-882b-4645-818e-1f69ce9e68f3 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:05.117432 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1101/2234] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:29:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 06:29:05.146878 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:05.154333 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1134/2235] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:29:04 2022] PUT /identity/v3/projects/c50588dbcfd54bb0b28556064c9b2d50/users/73432fdcfb0446dcb6e43e2e56a14ca2/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:29:05.158800 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5eadfc-882b-4645-818e-1f69ce9e68f3 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:05.170183 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5eadfc-882b-4645-818e-1f69ce9e68f3 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:05.240162 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99381f35-d04b-4927-9f0a-6b95e04d0491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:05.240900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99381f35-d04b-4927-9f0a-6b95e04d0491 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:05.241198 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99381f35-d04b-4927-9f0a-6b95e04d0491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:05.241493 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99381f35-d04b-4927-9f0a-6b95e04d0491 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:05.242310 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99381f35-d04b-4927-9f0a-6b95e04d0491 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:05.244260 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99381f35-d04b-4927-9f0a-6b95e04d0491 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:05.256071 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1102/2236] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:29:05.260641 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364c458-c70c-4bc9-a6c9-86297efb0d16 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:05.271719 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364c458-c70c-4bc9-a6c9-86297efb0d16 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:05.277194 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552c6f02-66b5-4d97-b020-6f68525488cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:05.278149 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c6f02-66b5-4d97-b020-6f68525488cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:05.278444 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c6f02-66b5-4d97-b020-6f68525488cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:05.278730 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c6f02-66b5-4d97-b020-6f68525488cf admin admin] PATH_INFO: `/v3/projects/c50588dbcfd54bb0b28556064c9b2d50/users/73432fdcfb0446dcb6e43e2e56a14ca2/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:05.282874 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552c6f02-66b5-4d97-b020-6f68525488cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c50588dbcfd54bb0b28556064c9b2d50, user_id=73432fdcfb0446dcb6e43e2e56a14ca2, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:05.294798 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552c6f02-66b5-4d97-b020-6f68525488cf admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:05.298979 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1135/2237] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:29:05 2022] PUT /identity/v3/projects/c50588dbcfd54bb0b28556064c9b2d50/users/73432fdcfb0446dcb6e43e2e56a14ca2/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:29:05.310075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233832da-8240-4dac-8246-53eb05529107 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:05.310364 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233832da-8240-4dac-8246-53eb05529107 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:05.310646 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233832da-8240-4dac-8246-53eb05529107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:05.342408 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-233832da-8240-4dac-8246-53eb05529107 None None] MFA Rules not processed for user `73432fdcfb0446dcb6e43e2e56a14ca2`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:05.343974 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-233832da-8240-4dac-8246-53eb05529107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:05.441035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1103/2238] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:05 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:29:05.455738 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552c6f02-66b5-4d97-b020-6f68525488cf admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:05.458731 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-552c6f02-66b5-4d97-b020-6f68525488cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:05.541733 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552c6f02-66b5-4d97-b020-6f68525488cf admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:05.545750 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee8068-6c92-4f5c-919d-f83c71bc37b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:05.547081 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee8068-6c92-4f5c-919d-f83c71bc37b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:05.547531 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee8068-6c92-4f5c-919d-f83c71bc37b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:05.547827 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee8068-6c92-4f5c-919d-f83c71bc37b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:05.549833 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e3ee8068-6c92-4f5c-919d-f83c71bc37b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:05.552574 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:05.629826 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ee8068-6c92-4f5c-919d-f83c71bc37b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:05.631434 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ee8068-6c92-4f5c-919d-f83c71bc37b4 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:05.644950 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1136/2239] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:29:05 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:29:06.011067 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233832da-8240-4dac-8246-53eb05529107 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:06.012706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-233832da-8240-4dac-8246-53eb05529107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:06.099507 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233832da-8240-4dac-8246-53eb05529107 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:06.104654 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f916036d-59a4-4f1f-8a6a-0c2d2aca768a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:06.106279 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916036d-59a4-4f1f-8a6a-0c2d2aca768a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:06.106629 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916036d-59a4-4f1f-8a6a-0c2d2aca768a admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:06.106983 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916036d-59a4-4f1f-8a6a-0c2d2aca768a admin admin] PATH_INFO: `/v3/projects/8031b2e212154353a53ea00aeb6b6eea` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:06.108933 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f916036d-59a4-4f1f-8a6a-0c2d2aca768a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8031b2e212154353a53ea00aeb6b6eea)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:06.112291 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f916036d-59a4-4f1f-8a6a-0c2d2aca768a admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:06.245315 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee8068-6c92-4f5c-919d-f83c71bc37b4 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:06.246943 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e3ee8068-6c92-4f5c-919d-f83c71bc37b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:06.270502 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee8068-6c92-4f5c-919d-f83c71bc37b4 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:06.279377 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f916036d-59a4-4f1f-8a6a-0c2d2aca768a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8031b2e212154353a53ea00aeb6b6eea', 'request_id': 'req-f916036d-59a4-4f1f-8a6a-0c2d2aca768a'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:06.288147 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f916036d-59a4-4f1f-8a6a-0c2d2aca768a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8031b2e212154353a53ea00aeb6b6eea', 'request_id': 'req-f916036d-59a4-4f1f-8a6a-0c2d2aca768a'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:06.366527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349379fc-0b96-4a2c-9c24-6dec32e5becc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:06.367621 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349379fc-0b96-4a2c-9c24-6dec32e5becc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:06.367988 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349379fc-0b96-4a2c-9c24-6dec32e5becc admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:06.368236 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349379fc-0b96-4a2c-9c24-6dec32e5becc admin admin] PATH_INFO: `/v3/users/519f82e56dec4aed9bafa7dea6a436fb` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:06.369754 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349379fc-0b96-4a2c-9c24-6dec32e5becc admin admin] RBAC: Authorizing `identity:delete_user(user_id=519f82e56dec4aed9bafa7dea6a436fb)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:06.373198 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349379fc-0b96-4a2c-9c24-6dec32e5becc admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:06.380998 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f916036d-59a4-4f1f-8a6a-0c2d2aca768a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8031b2e212154353a53ea00aeb6b6eea', 'request_id': 'req-f916036d-59a4-4f1f-8a6a-0c2d2aca768a'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:06.402087 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f916036d-59a4-4f1f-8a6a-0c2d2aca768a admin admin] The token cache is being invalidate because project 8031b2e212154353a53ea00aeb6b6eea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:29:06.402087 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f916036d-59a4-4f1f-8a6a-0c2d2aca768a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:06.403975 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f916036d-59a4-4f1f-8a6a-0c2d2aca768a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:06.413505 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1104/2240] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:29:06 2022] DELETE /identity/v3/projects/8031b2e212154353a53ea00aeb6b6eea => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:29:06.419692 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1c101-5ba0-4981-bec4-ec0e2be5999d None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:06.419692 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1c101-5ba0-4981-bec4-ec0e2be5999d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:06.419692 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1c101-5ba0-4981-bec4-ec0e2be5999d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:06.439119 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-2ba1c101-5ba0-4981-bec4-ec0e2be5999d None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:06.440523 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-2ba1c101-5ba0-4981-bec4-ec0e2be5999d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:06.451844 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1105/2241] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:29:06.463015 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f0664-a294-4958-9f24-7050b49e9ab7 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:06.463375 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f0664-a294-4958-9f24-7050b49e9ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:06.463696 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f0664-a294-4958-9f24-7050b49e9ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:06.484320 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-374f0664-a294-4958-9f24-7050b49e9ab7 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:06.485535 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-374f0664-a294-4958-9f24-7050b49e9ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:06.507599 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1106/2242] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:29:06.509479 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f0664-a294-4958-9f24-7050b49e9ab7 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:06.514598 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:06.522036 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f0664-a294-4958-9f24-7050b49e9ab7 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:06.527717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37956fa-f017-42d2-8a73-820284ddeabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:06.528539 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37956fa-f017-42d2-8a73-820284ddeabd admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:06.528830 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37956fa-f017-42d2-8a73-820284ddeabd admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:06.529118 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37956fa-f017-42d2-8a73-820284ddeabd admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:06.529950 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37956fa-f017-42d2-8a73-820284ddeabd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:29:06.530261 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37956fa-f017-42d2-8a73-820284ddeabd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:06.532054 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37956fa-f017-42d2-8a73-820284ddeabd admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:06.547521 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1107/2243] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:29:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:29:06.552560 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37956fa-f017-42d2-8a73-820284ddeabd admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:06.566060 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37956fa-f017-42d2-8a73-820284ddeabd admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:06.571531 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c03cd5-c18e-4de8-9cc7-a1681c7f8ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:06.572218 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c03cd5-c18e-4de8-9cc7-a1681c7f8ade admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:06.572561 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c03cd5-c18e-4de8-9cc7-a1681c7f8ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:06.572848 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c03cd5-c18e-4de8-9cc7-a1681c7f8ade admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:06.574031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c03cd5-c18e-4de8-9cc7-a1681c7f8ade admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:06.576049 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c03cd5-c18e-4de8-9cc7-a1681c7f8ade admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:06.576579 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:06.598539 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-349379fc-0b96-4a2c-9c24-6dec32e5becc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '519f82e56dec4aed9bafa7dea6a436fb'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:06.607886 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-349379fc-0b96-4a2c-9c24-6dec32e5becc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '519f82e56dec4aed9bafa7dea6a436fb'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:06.609975 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-349379fc-0b96-4a2c-9c24-6dec32e5becc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '519f82e56dec4aed9bafa7dea6a436fb'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:06.610650 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-349379fc-0b96-4a2c-9c24-6dec32e5becc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '519f82e56dec4aed9bafa7dea6a436fb'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:06.632348 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-349379fc-0b96-4a2c-9c24-6dec32e5becc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '519f82e56dec4aed9bafa7dea6a436fb'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:06.696862 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1108/2244] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:29:06 2022] POST /identity/v3/projects => generated 379 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:29:06.700708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c03cd5-c18e-4de8-9cc7-a1681c7f8ade admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:06.703914 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-65c03cd5-c18e-4de8-9cc7-a1681c7f8ade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:06.733431 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1137/2245] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:29:06 2022] DELETE /identity/v3/users/519f82e56dec4aed9bafa7dea6a436fb => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:29:06.735168 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e3f871-aff6-4e25-be35-1630bc0ee56c None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:06.735477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e3f871-aff6-4e25-be35-1630bc0ee56c None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:06.737053 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e3f871-aff6-4e25-be35-1630bc0ee56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:06.762952 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-05e3f871-aff6-4e25-be35-1630bc0ee56c None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:06.766002 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-05e3f871-aff6-4e25-be35-1630bc0ee56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:06.861615 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c03cd5-c18e-4de8-9cc7-a1681c7f8ade admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:06.885877 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1138/2246] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:29:06.895087 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e3f871-aff6-4e25-be35-1630bc0ee56c None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:06.897697 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-05e3f871-aff6-4e25-be35-1630bc0ee56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:06.968680 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aae07f-68d2-4a2e-b2d6-1ab5a151d254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:06.970153 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aae07f-68d2-4a2e-b2d6-1ab5a151d254 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:06.971218 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aae07f-68d2-4a2e-b2d6-1ab5a151d254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:06.971708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aae07f-68d2-4a2e-b2d6-1ab5a151d254 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:06.973128 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aae07f-68d2-4a2e-b2d6-1ab5a151d254 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:06.976291 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aae07f-68d2-4a2e-b2d6-1ab5a151d254 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:06.982671 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e3f871-aff6-4e25-be35-1630bc0ee56c None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:06.988825 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea294bc3-d58e-46b3-b9b8-d525dd5420e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:06.989845 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea294bc3-d58e-46b3-b9b8-d525dd5420e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:06.989845 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea294bc3-d58e-46b3-b9b8-d525dd5420e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:06.990039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea294bc3-d58e-46b3-b9b8-d525dd5420e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:07.004217 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea294bc3-d58e-46b3-b9b8-d525dd5420e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:07.006116 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea294bc3-d58e-46b3-b9b8-d525dd5420e9 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:07.031294 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1139/2247] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:29:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 06:29:07.115221 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1109/2248] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:29:06 2022] POST /identity/v3/users => generated 290 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:29:07.119691 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea294bc3-d58e-46b3-b9b8-d525dd5420e9 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:07.132036 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea294bc3-d58e-46b3-b9b8-d525dd5420e9 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:07.141299 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d30a7d4-4130-4b3b-8ac7-68afca7b8fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:07.142059 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30a7d4-4130-4b3b-8ac7-68afca7b8fab admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:07.142355 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30a7d4-4130-4b3b-8ac7-68afca7b8fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:07.142641 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30a7d4-4130-4b3b-8ac7-68afca7b8fab admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:07.143396 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d30a7d4-4130-4b3b-8ac7-68afca7b8fab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:07.145159 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d30a7d4-4130-4b3b-8ac7-68afca7b8fab admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:07.156479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1140/2249] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:29:07.160741 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aae07f-68d2-4a2e-b2d6-1ab5a151d254 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:07.172584 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aae07f-68d2-4a2e-b2d6-1ab5a151d254 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:07.177503 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e073991e-1597-4a23-9f51-5ee886ff6616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:07.178173 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073991e-1597-4a23-9f51-5ee886ff6616 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:07.178457 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073991e-1597-4a23-9f51-5ee886ff6616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:07.178735 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073991e-1597-4a23-9f51-5ee886ff6616 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:07.179481 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e073991e-1597-4a23-9f51-5ee886ff6616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:07.181121 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e073991e-1597-4a23-9f51-5ee886ff6616 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:07.196957 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1110/2250] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 06:29:07.201680 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d30a7d4-4130-4b3b-8ac7-68afca7b8fab admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:07.219489 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d30a7d4-4130-4b3b-8ac7-68afca7b8fab admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:07.224484 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16435bc-916d-49c3-bbb0-1069fdeb96e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:07.225541 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16435bc-916d-49c3-bbb0-1069fdeb96e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:07.225881 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16435bc-916d-49c3-bbb0-1069fdeb96e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:07.226168 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16435bc-916d-49c3-bbb0-1069fdeb96e1 admin admin] PATH_INFO: `/v3/projects/fbaf0edf7bae44dfb37603b5648ecb64/users/621a7ea4f60545d5a5c30b2b7a370bf0/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:07.243402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16435bc-916d-49c3-bbb0-1069fdeb96e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbaf0edf7bae44dfb37603b5648ecb64, user_id=621a7ea4f60545d5a5c30b2b7a370bf0, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:07.246362 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16435bc-916d-49c3-bbb0-1069fdeb96e1 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:07.432340 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1141/2251] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:29:07 2022] PUT /identity/v3/projects/fbaf0edf7bae44dfb37603b5648ecb64/users/621a7ea4f60545d5a5c30b2b7a370bf0/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:29:07.435268 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db9317-a6fa-4af0-9190-a19c15086fb4 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:07.435928 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db9317-a6fa-4af0-9190-a19c15086fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:07.436266 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db9317-a6fa-4af0-9190-a19c15086fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:07.456336 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-57db9317-a6fa-4af0-9190-a19c15086fb4 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:07.457699 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-57db9317-a6fa-4af0-9190-a19c15086fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:07.535852 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1111/2252] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:29:07.549412 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:07.553237 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16435bc-916d-49c3-bbb0-1069fdeb96e1 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:07.615784 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:07.638980 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16435bc-916d-49c3-bbb0-1069fdeb96e1 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:07.644340 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09e2d79-9e84-4566-8ce9-f05c075d6910 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:07.645227 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e2d79-9e84-4566-8ce9-f05c075d6910 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:07.645967 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e2d79-9e84-4566-8ce9-f05c075d6910 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:07.647136 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e2d79-9e84-4566-8ce9-f05c075d6910 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:07.659901 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09e2d79-9e84-4566-8ce9-f05c075d6910 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:07.661568 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09e2d79-9e84-4566-8ce9-f05c075d6910 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:07.677922 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1142/2253] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:29:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:29:07.723248 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57db9317-a6fa-4af0-9190-a19c15086fb4 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:07.723812 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-57db9317-a6fa-4af0-9190-a19c15086fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:07.745650 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57db9317-a6fa-4af0-9190-a19c15086fb4 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:07.754365 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9900ef-dc92-404d-ba16-3107ec5ba8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:07.757541 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9900ef-dc92-404d-ba16-3107ec5ba8e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:07.757931 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9900ef-dc92-404d-ba16-3107ec5ba8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:07.758267 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9900ef-dc92-404d-ba16-3107ec5ba8e0 admin admin] PATH_INFO: `/v3/projects/575ddb99e9eb4fcf8b0c8cc1ac574616` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:07.760059 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9900ef-dc92-404d-ba16-3107ec5ba8e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=575ddb99e9eb4fcf8b0c8cc1ac574616)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:07.762603 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9900ef-dc92-404d-ba16-3107ec5ba8e0 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:07.951960 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-2e9900ef-dc92-404d-ba16-3107ec5ba8e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '575ddb99e9eb4fcf8b0c8cc1ac574616', 'request_id': 'req-2e9900ef-dc92-404d-ba16-3107ec5ba8e0'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:07.959770 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-2e9900ef-dc92-404d-ba16-3107ec5ba8e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '575ddb99e9eb4fcf8b0c8cc1ac574616', 'request_id': 'req-2e9900ef-dc92-404d-ba16-3107ec5ba8e0'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:08.038383 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-2e9900ef-dc92-404d-ba16-3107ec5ba8e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '575ddb99e9eb4fcf8b0c8cc1ac574616', 'request_id': 'req-2e9900ef-dc92-404d-ba16-3107ec5ba8e0'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:08.059724 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-2e9900ef-dc92-404d-ba16-3107ec5ba8e0 admin admin] The token cache is being invalidate because project 575ddb99e9eb4fcf8b0c8cc1ac574616 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:29:08.060177 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-2e9900ef-dc92-404d-ba16-3107ec5ba8e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:08.061173 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-2e9900ef-dc92-404d-ba16-3107ec5ba8e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:08.064112 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1112/2254] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:29:07 2022] DELETE /identity/v3/projects/575ddb99e9eb4fcf8b0c8cc1ac574616 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:29:08.077758 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9895df-93fe-4d72-a65a-79ca4cdd4ae0 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:08.079815 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9895df-93fe-4d72-a65a-79ca4cdd4ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:08.080432 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9895df-93fe-4d72-a65a-79ca4cdd4ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:08.131583 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-0f9895df-93fe-4d72-a65a-79ca4cdd4ae0 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:08.133063 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-0f9895df-93fe-4d72-a65a-79ca4cdd4ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:08.211428 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1143/2255] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:29:08.217947 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393cf203-95b0-42e2-a65f-4ed91024dc7f None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:08.219274 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393cf203-95b0-42e2-a65f-4ed91024dc7f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:08.219274 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393cf203-95b0-42e2-a65f-4ed91024dc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:08.240246 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-393cf203-95b0-42e2-a65f-4ed91024dc7f None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:08.241620 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-393cf203-95b0-42e2-a65f-4ed91024dc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:08.251104 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1113/2256] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:29:08.255233 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9895df-93fe-4d72-a65a-79ca4cdd4ae0 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:08.265282 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9895df-93fe-4d72-a65a-79ca4cdd4ae0 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:08.270244 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50678e-a2db-471a-bcff-d8b1878a6068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:08.271018 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50678e-a2db-471a-bcff-d8b1878a6068 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:08.271305 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50678e-a2db-471a-bcff-d8b1878a6068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:08.271693 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50678e-a2db-471a-bcff-d8b1878a6068 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:08.272387 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa50678e-a2db-471a-bcff-d8b1878a6068 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:29:08.272766 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa50678e-a2db-471a-bcff-d8b1878a6068 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:08.274546 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa50678e-a2db-471a-bcff-d8b1878a6068 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:08.290656 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1144/2257] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:29:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:29:08.295104 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393cf203-95b0-42e2-a65f-4ed91024dc7f None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:08.306919 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393cf203-95b0-42e2-a65f-4ed91024dc7f None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:08.313931 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd2f16-af5f-43fc-a4ec-ce06632c0c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:08.315634 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd2f16-af5f-43fc-a4ec-ce06632c0c1a admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:08.315634 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd2f16-af5f-43fc-a4ec-ce06632c0c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:08.315634 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd2f16-af5f-43fc-a4ec-ce06632c0c1a admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:08.315991 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cd2f16-af5f-43fc-a4ec-ce06632c0c1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:08.318024 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cd2f16-af5f-43fc-a4ec-ce06632c0c1a admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:08.502971 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1114/2258] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:29:08 2022] POST /identity/v3/projects => generated 389 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:29:08.508551 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50678e-a2db-471a-bcff-d8b1878a6068 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:08.521582 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50678e-a2db-471a-bcff-d8b1878a6068 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:08.592459 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78bdb2-6fd5-4665-83c1-3299f0421cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:08.593597 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78bdb2-6fd5-4665-83c1-3299f0421cb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:08.593916 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78bdb2-6fd5-4665-83c1-3299f0421cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:08.594205 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78bdb2-6fd5-4665-83c1-3299f0421cb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:08.595037 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b78bdb2-6fd5-4665-83c1-3299f0421cb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:08.597141 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b78bdb2-6fd5-4665-83c1-3299f0421cb7 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:08.667156 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1145/2259] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:29:08 2022] POST /identity/v3/users => generated 295 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:29:08.672891 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd2f16-af5f-43fc-a4ec-ce06632c0c1a admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:08.691981 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd2f16-af5f-43fc-a4ec-ce06632c0c1a admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:08.697695 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c62fc90-1bba-4919-abcd-882efb6f2cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:08.698441 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c62fc90-1bba-4919-abcd-882efb6f2cb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:08.698736 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c62fc90-1bba-4919-abcd-882efb6f2cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:08.699015 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c62fc90-1bba-4919-abcd-882efb6f2cb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:08.699763 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c62fc90-1bba-4919-abcd-882efb6f2cb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:08.701573 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c62fc90-1bba-4919-abcd-882efb6f2cb3 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:08.728379 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1115/2260] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:29:08 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:29:08.732449 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78bdb2-6fd5-4665-83c1-3299f0421cb7 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:08.748122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78bdb2-6fd5-4665-83c1-3299f0421cb7 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:08.756656 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd9df2-0ef3-475a-86d3-262a8a799350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:08.757371 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd9df2-0ef3-475a-86d3-262a8a799350 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:08.758032 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd9df2-0ef3-475a-86d3-262a8a799350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:08.758583 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd9df2-0ef3-475a-86d3-262a8a799350 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:08.760299 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bd9df2-0ef3-475a-86d3-262a8a799350 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:08.762231 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bd9df2-0ef3-475a-86d3-262a8a799350 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:08.778566 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1146/2261] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:29:08 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:29:08.783236 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c62fc90-1bba-4919-abcd-882efb6f2cb3 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:08.797444 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c62fc90-1bba-4919-abcd-882efb6f2cb3 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:08.807433 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b19664-4ca8-4a94-8a0b-3f1d69076022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:08.808665 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b19664-4ca8-4a94-8a0b-3f1d69076022 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:08.809197 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b19664-4ca8-4a94-8a0b-3f1d69076022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:08.810698 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b19664-4ca8-4a94-8a0b-3f1d69076022 admin admin] PATH_INFO: `/v3/projects/3669fc8553d94dd589e027a5d772c8c7/users/bbefc8439c9340cbb30262891a0ce66b/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:08.844897 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b19664-4ca8-4a94-8a0b-3f1d69076022 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3669fc8553d94dd589e027a5d772c8c7, user_id=bbefc8439c9340cbb30262891a0ce66b, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:08.848454 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b19664-4ca8-4a94-8a0b-3f1d69076022 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:09.012476 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1116/2262] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:29:08 2022] PUT /identity/v3/projects/3669fc8553d94dd589e027a5d772c8c7/users/bbefc8439c9340cbb30262891a0ce66b/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:29:09.013432 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5362fd-d0a5-416b-9e10-331c066983b6 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:09.013730 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5362fd-d0a5-416b-9e10-331c066983b6 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:09.014018 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5362fd-d0a5-416b-9e10-331c066983b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:09.055692 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-8c5362fd-d0a5-416b-9e10-331c066983b6 None None] MFA Rules not processed for user `bbefc8439c9340cbb30262891a0ce66b`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:09.057361 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-8c5362fd-d0a5-416b-9e10-331c066983b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:09.092501 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:09.111590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b19664-4ca8-4a94-8a0b-3f1d69076022 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:09.114519 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-09b19664-4ca8-4a94-8a0b-3f1d69076022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:09.164509 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1147/2263] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:29:09.184260 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5362fd-d0a5-416b-9e10-331c066983b6 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:09.185960 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-8c5362fd-d0a5-416b-9e10-331c066983b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:09.202643 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b19664-4ca8-4a94-8a0b-3f1d69076022 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:09.205148 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5362fd-d0a5-416b-9e10-331c066983b6 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:09.208591 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d353e-9f78-47e8-b2d1-56b77cacc8b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:09.209082 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119a019-7094-4c5b-989d-f80c4d4066b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:09.209473 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d353e-9f78-47e8-b2d1-56b77cacc8b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:09.209844 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d353e-9f78-47e8-b2d1-56b77cacc8b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:09.210078 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119a019-7094-4c5b-989d-f80c4d4066b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:09.210291 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119a019-7094-4c5b-989d-f80c4d4066b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:09.210456 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d353e-9f78-47e8-b2d1-56b77cacc8b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:09.210661 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119a019-7094-4c5b-989d-f80c4d4066b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:09.212737 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-377d353e-9f78-47e8-b2d1-56b77cacc8b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:09.220898 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f119a019-7094-4c5b-989d-f80c4d4066b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:09.222488 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f119a019-7094-4c5b-989d-f80c4d4066b4 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:09.244089 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1148/2264] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:29:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 06:29:09.248568 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:09.293776 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d353e-9f78-47e8-b2d1-56b77cacc8b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:09.295406 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d353e-9f78-47e8-b2d1-56b77cacc8b2 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:09.307739 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1117/2265] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:29:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:29:11.319514 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119a019-7094-4c5b-989d-f80c4d4066b4 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:11.330130 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119a019-7094-4c5b-989d-f80c4d4066b4 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:11.335523 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81fef34-0ea2-44d7-999a-6f428d5d5d4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:11.336273 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81fef34-0ea2-44d7-999a-6f428d5d5d4c service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:11.336661 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81fef34-0ea2-44d7-999a-6f428d5d5d4c service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:11.337010 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81fef34-0ea2-44d7-999a-6f428d5d5d4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:11.339073 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c81fef34-0ea2-44d7-999a-6f428d5d5d4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:11.429042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81fef34-0ea2-44d7-999a-6f428d5d5d4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:11.431658 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81fef34-0ea2-44d7-999a-6f428d5d5d4c service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:11.449103 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1149/2266] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:29:11 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:29:14.543007 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf91d1-4ccc-41f7-87da-eb850193d7d6 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:14.544142 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf91d1-4ccc-41f7-87da-eb850193d7d6 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:14.545149 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf91d1-4ccc-41f7-87da-eb850193d7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:14.589302 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-44cf91d1-4ccc-41f7-87da-eb850193d7d6 None None] MFA Rules not processed for user `621a7ea4f60545d5a5c30b2b7a370bf0`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:14.590715 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-44cf91d1-4ccc-41f7-87da-eb850193d7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:14.717193 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1118/2267] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:29:14.723286 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81fef34-0ea2-44d7-999a-6f428d5d5d4c service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:14.725894 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c81fef34-0ea2-44d7-999a-6f428d5d5d4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:14.809059 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81fef34-0ea2-44d7-999a-6f428d5d5d4c service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:14.816439 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef3141a-ee3f-4977-9e69-5d83c0d7c8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:14.817331 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3141a-ee3f-4977-9e69-5d83c0d7c8cb admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:14.817740 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3141a-ee3f-4977-9e69-5d83c0d7c8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:14.818108 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3141a-ee3f-4977-9e69-5d83c0d7c8cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:14.818973 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef3141a-ee3f-4977-9e69-5d83c0d7c8cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:14.821209 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef3141a-ee3f-4977-9e69-5d83c0d7c8cb admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:14.917948 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1150/2268] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:29:14 2022] POST /identity/v3/projects => generated 377 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:29:14.925729 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf91d1-4ccc-41f7-87da-eb850193d7d6 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:14.943371 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf91d1-4ccc-41f7-87da-eb850193d7d6 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:15.045954 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde000a-bf15-41e8-aa67-6cdee7ea466a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:15.047668 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde000a-bf15-41e8-aa67-6cdee7ea466a admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:15.048161 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde000a-bf15-41e8-aa67-6cdee7ea466a admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:15.049065 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde000a-bf15-41e8-aa67-6cdee7ea466a admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:15.049271 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cde000a-bf15-41e8-aa67-6cdee7ea466a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:15.051257 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cde000a-bf15-41e8-aa67-6cdee7ea466a admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:15.141443 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1119/2269] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:29:14 2022] POST /identity/v3/users => generated 295 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:29:15.147061 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef3141a-ee3f-4977-9e69-5d83c0d7c8cb admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:15.171898 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef3141a-ee3f-4977-9e69-5d83c0d7c8cb admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:15.177524 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c42869-a038-4b11-ad4a-3230cd4c2aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:15.178224 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c42869-a038-4b11-ad4a-3230cd4c2aac admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:15.179419 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c42869-a038-4b11-ad4a-3230cd4c2aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:15.179419 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c42869-a038-4b11-ad4a-3230cd4c2aac admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:15.179886 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c42869-a038-4b11-ad4a-3230cd4c2aac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:15.181502 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c42869-a038-4b11-ad4a-3230cd4c2aac admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:15.196073 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1151/2270] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:29:15.200565 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde000a-bf15-41e8-aa67-6cdee7ea466a admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:15.211729 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde000a-bf15-41e8-aa67-6cdee7ea466a admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:15.217487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571482b0-bc5f-497a-a565-64d3aebc5d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:15.219158 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571482b0-bc5f-497a-a565-64d3aebc5d47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:15.219486 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571482b0-bc5f-497a-a565-64d3aebc5d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:15.219816 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571482b0-bc5f-497a-a565-64d3aebc5d47 admin admin] PATH_INFO: `/v3/projects/688a4570d2ed42869afcf8cb84f6af89/users/2dc107028e2e4457b1e862669fb1d876/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:15.239563 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571482b0-bc5f-497a-a565-64d3aebc5d47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=688a4570d2ed42869afcf8cb84f6af89, user_id=2dc107028e2e4457b1e862669fb1d876, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:15.242768 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571482b0-bc5f-497a-a565-64d3aebc5d47 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:15.315082 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1120/2271] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:29:15 2022] PUT /identity/v3/projects/688a4570d2ed42869afcf8cb84f6af89/users/2dc107028e2e4457b1e862669fb1d876/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:29:15.319780 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c42869-a038-4b11-ad4a-3230cd4c2aac admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:15.332301 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c42869-a038-4b11-ad4a-3230cd4c2aac admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:15.340778 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:15.412196 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e1454-fa58-468d-807b-2d88f911bee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:15.412919 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e1454-fa58-468d-807b-2d88f911bee4 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:15.413212 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e1454-fa58-468d-807b-2d88f911bee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:15.413776 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e1454-fa58-468d-807b-2d88f911bee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:15.414768 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802e1454-fa58-468d-807b-2d88f911bee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:15.416856 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802e1454-fa58-468d-807b-2d88f911bee4 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:15.429678 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1152/2272] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:29:15.433985 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571482b0-bc5f-497a-a565-64d3aebc5d47 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:15.444904 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571482b0-bc5f-497a-a565-64d3aebc5d47 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:15.450622 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d6e3a0-bc60-4653-b4b0-049e1aae98b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:15.451673 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6e3a0-bc60-4653-b4b0-049e1aae98b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:15.452061 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6e3a0-bc60-4653-b4b0-049e1aae98b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:15.452521 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6e3a0-bc60-4653-b4b0-049e1aae98b4 admin admin] PATH_INFO: `/v3/projects/688a4570d2ed42869afcf8cb84f6af89/users/2dc107028e2e4457b1e862669fb1d876/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:15.455270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d6e3a0-bc60-4653-b4b0-049e1aae98b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=688a4570d2ed42869afcf8cb84f6af89, user_id=2dc107028e2e4457b1e862669fb1d876, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:15.458095 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d6e3a0-bc60-4653-b4b0-049e1aae98b4 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:15.471320 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1121/2273] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:29:15 2022] PUT /identity/v3/projects/688a4570d2ed42869afcf8cb84f6af89/users/2dc107028e2e4457b1e862669fb1d876/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:29:22.807976 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f21e4-089c-4e13-a2d2-2206b30b0168 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:22.807976 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f21e4-089c-4e13-a2d2-2206b30b0168 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:22.807976 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f21e4-089c-4e13-a2d2-2206b30b0168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:22.839045 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-e25f21e4-089c-4e13-a2d2-2206b30b0168 None None] MFA Rules not processed for user `2dc107028e2e4457b1e862669fb1d876`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:22.841466 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e25f21e4-089c-4e13-a2d2-2206b30b0168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:22.917962 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1153/2274] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:22 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:29:22.936108 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d6e3a0-bc60-4653-b4b0-049e1aae98b4 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:22.944867 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a8d6e3a0-bc60-4653-b4b0-049e1aae98b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:22.957499 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:23.057049 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d6e3a0-bc60-4653-b4b0-049e1aae98b4 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:23.062742 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296e37d-f77c-4c6b-8941-d1e4094d15d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:23.063334 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296e37d-f77c-4c6b-8941-d1e4094d15d4 service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:23.063623 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296e37d-f77c-4c6b-8941-d1e4094d15d4 service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:23.063910 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296e37d-f77c-4c6b-8941-d1e4094d15d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:23.108659 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e693d2-e127-4a09-864e-c0bc6274a152 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:23.108958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e693d2-e127-4a09-864e-c0bc6274a152 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:23.109244 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e693d2-e127-4a09-864e-c0bc6274a152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:23.126540 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2296e37d-f77c-4c6b-8941-d1e4094d15d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:23.127917 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-09e693d2-e127-4a09-864e-c0bc6274a152 None None] MFA Rules not processed for user `76f3dd56c5d24c7490e337b2aa90b584`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:23.128367 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2296e37d-f77c-4c6b-8941-d1e4094d15d4 service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:23.129269 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-09e693d2-e127-4a09-864e-c0bc6274a152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:23.141957 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1122/2275] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:29:22 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:29:23.195506 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1154/2276] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:29:25.490867 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296e37d-f77c-4c6b-8941-d1e4094d15d4 service nova] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:25.492551 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-2296e37d-f77c-4c6b-8941-d1e4094d15d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:25.632401 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296e37d-f77c-4c6b-8941-d1e4094d15d4 service nova] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:25.637223 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e693d2-e127-4a09-864e-c0bc6274a152 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:25.638233 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fba342-1d69-456e-8105-c11ab2e02c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:25.639284 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fba342-1d69-456e-8105-c11ab2e02c7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:25.639574 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fba342-1d69-456e-8105-c11ab2e02c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:25.640821 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fba342-1d69-456e-8105-c11ab2e02c7d admin admin] PATH_INFO: `/v3/users/87ce6e5f3976486fad9a6991e93590da` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:25.642838 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fba342-1d69-456e-8105-c11ab2e02c7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=87ce6e5f3976486fad9a6991e93590da)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:25.643150 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-09e693d2-e127-4a09-864e-c0bc6274a152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:25.645115 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fba342-1d69-456e-8105-c11ab2e02c7d admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:25.732130 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:25.786499 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e693d2-e127-4a09-864e-c0bc6274a152 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:25.793727 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62270cd1-5c8d-4649-bf86-f1b14527a13d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3560534427544cdabfd0ba702a0a165', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:25.795772 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62270cd1-5c8d-4649-bf86-f1b14527a13d service placement] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:25.797278 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62270cd1-5c8d-4649-bf86-f1b14527a13d service placement] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:25.797769 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62270cd1-5c8d-4649-bf86-f1b14527a13d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:25.800515 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-62270cd1-5c8d-4649-bf86-f1b14527a13d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:25.819198 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62270cd1-5c8d-4649-bf86-f1b14527a13d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:25.822927 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62270cd1-5c8d-4649-bf86-f1b14527a13d service placement] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:25.840896 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1155/2277] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:29:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:29:25.930060 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f9fba342-1d69-456e-8105-c11ab2e02c7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87ce6e5f3976486fad9a6991e93590da'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:25.936292 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:25.966780 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f9fba342-1d69-456e-8105-c11ab2e02c7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87ce6e5f3976486fad9a6991e93590da'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:26.022182 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f9fba342-1d69-456e-8105-c11ab2e02c7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87ce6e5f3976486fad9a6991e93590da'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:26.023049 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f9fba342-1d69-456e-8105-c11ab2e02c7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87ce6e5f3976486fad9a6991e93590da'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:26.023744 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f9fba342-1d69-456e-8105-c11ab2e02c7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87ce6e5f3976486fad9a6991e93590da'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:26.189322 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1123/2278] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:29:25 2022] DELETE /identity/v3/users/87ce6e5f3976486fad9a6991e93590da => generated 0 bytes in 699 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:29:26.197354 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5115086a-aa98-47b0-be5e-79fb9f0c49f7 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:26.197354 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5115086a-aa98-47b0-be5e-79fb9f0c49f7 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:26.197669 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5115086a-aa98-47b0-be5e-79fb9f0c49f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:26.232328 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-5115086a-aa98-47b0-be5e-79fb9f0c49f7 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:26.234748 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-5115086a-aa98-47b0-be5e-79fb9f0c49f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:26.359993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1156/2279] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:29:26.374452 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fba342-1d69-456e-8105-c11ab2e02c7d admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:26.376728 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f9fba342-1d69-456e-8105-c11ab2e02c7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:26.521642 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fba342-1d69-456e-8105-c11ab2e02c7d admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:26.526630 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9875f2ce-fd8d-4020-a010-add96fffbfd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:26.527312 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875f2ce-fd8d-4020-a010-add96fffbfd0 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:26.527767 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875f2ce-fd8d-4020-a010-add96fffbfd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:26.527918 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875f2ce-fd8d-4020-a010-add96fffbfd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:26.541294 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9875f2ce-fd8d-4020-a010-add96fffbfd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:26.543840 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9875f2ce-fd8d-4020-a010-add96fffbfd0 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:26.567866 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1124/2280] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:29:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:29:26.682186 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5115086a-aa98-47b0-be5e-79fb9f0c49f7 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:26.684785 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-5115086a-aa98-47b0-be5e-79fb9f0c49f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:26.708981 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5115086a-aa98-47b0-be5e-79fb9f0c49f7 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:26.719046 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b7852-e9df-460a-a491-b1f1ff2d52c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:26.720511 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b7852-e9df-460a-a491-b1f1ff2d52c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:26.720894 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b7852-e9df-460a-a491-b1f1ff2d52c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:26.721193 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b7852-e9df-460a-a491-b1f1ff2d52c1 admin admin] PATH_INFO: `/v3/users/ef02ea3b60ca4466924379e36d09df0e` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:26.722666 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b7852-e9df-460a-a491-b1f1ff2d52c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef02ea3b60ca4466924379e36d09df0e)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:26.724925 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b7852-e9df-460a-a491-b1f1ff2d52c1 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:26.963918 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-373b7852-e9df-460a-a491-b1f1ff2d52c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef02ea3b60ca4466924379e36d09df0e'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:26.978715 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-373b7852-e9df-460a-a491-b1f1ff2d52c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef02ea3b60ca4466924379e36d09df0e'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:26.980091 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-373b7852-e9df-460a-a491-b1f1ff2d52c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef02ea3b60ca4466924379e36d09df0e'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:26.981273 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-373b7852-e9df-460a-a491-b1f1ff2d52c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef02ea3b60ca4466924379e36d09df0e'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:27.002490 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-373b7852-e9df-460a-a491-b1f1ff2d52c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef02ea3b60ca4466924379e36d09df0e'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:27.119996 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1157/2281] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:29:26 2022] DELETE /identity/v3/users/ef02ea3b60ca4466924379e36d09df0e => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:29:27.125820 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3590e14-11ad-48b5-8869-619c5fdf57fb None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:27.126112 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3590e14-11ad-48b5-8869-619c5fdf57fb None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:27.126423 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3590e14-11ad-48b5-8869-619c5fdf57fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:27.147300 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-a3590e14-11ad-48b5-8869-619c5fdf57fb None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:27.148909 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a3590e14-11ad-48b5-8869-619c5fdf57fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:27.205630 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b7852-e9df-460a-a491-b1f1ff2d52c1 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:27.207109 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-373b7852-e9df-460a-a491-b1f1ff2d52c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:27.249200 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1125/2282] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:29:27.259867 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3590e14-11ad-48b5-8869-619c5fdf57fb None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:27.261910 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a3590e14-11ad-48b5-8869-619c5fdf57fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:27.269698 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:27.295678 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b7852-e9df-460a-a491-b1f1ff2d52c1 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:27.302715 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea769eb-d203-4889-be97-9569569ba81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:27.303586 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea769eb-d203-4889-be97-9569569ba81d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:27.303872 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea769eb-d203-4889-be97-9569569ba81d admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:27.304151 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea769eb-d203-4889-be97-9569569ba81d admin admin] PATH_INFO: `/v3/projects/3515fcef23ba4c7f9224404a96dc1b7c` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:27.305559 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea769eb-d203-4889-be97-9569569ba81d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3515fcef23ba4c7f9224404a96dc1b7c)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:27.309118 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea769eb-d203-4889-be97-9569569ba81d admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:27.343532 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3590e14-11ad-48b5-8869-619c5fdf57fb None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:27.347917 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84fb3ea-4d6d-415f-8139-f29f087e721f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:27.348620 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84fb3ea-4d6d-415f-8139-f29f087e721f service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:27.348912 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84fb3ea-4d6d-415f-8139-f29f087e721f service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:27.349193 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84fb3ea-4d6d-415f-8139-f29f087e721f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:27.360727 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84fb3ea-4d6d-415f-8139-f29f087e721f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:27.362176 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84fb3ea-4d6d-415f-8139-f29f087e721f service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:27.376735 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1126/2283] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:29:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 06:29:27.450830 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8ea769eb-d203-4889-be97-9569569ba81d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3515fcef23ba4c7f9224404a96dc1b7c', 'request_id': 'req-8ea769eb-d203-4889-be97-9569569ba81d'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:27.460977 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8ea769eb-d203-4889-be97-9569569ba81d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3515fcef23ba4c7f9224404a96dc1b7c', 'request_id': 'req-8ea769eb-d203-4889-be97-9569569ba81d'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:27.558686 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8ea769eb-d203-4889-be97-9569569ba81d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3515fcef23ba4c7f9224404a96dc1b7c', 'request_id': 'req-8ea769eb-d203-4889-be97-9569569ba81d'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:27.573905 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8ea769eb-d203-4889-be97-9569569ba81d admin admin] The token cache is being invalidate because project 3515fcef23ba4c7f9224404a96dc1b7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:29:27.574012 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8ea769eb-d203-4889-be97-9569569ba81d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:27.575423 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8ea769eb-d203-4889-be97-9569569ba81d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:27.577426 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1158/2284] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:29:27 2022] DELETE /identity/v3/projects/3515fcef23ba4c7f9224404a96dc1b7c => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:29:27.581641 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84fb3ea-4d6d-415f-8139-f29f087e721f service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:27.592097 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84fb3ea-4d6d-415f-8139-f29f087e721f service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:27.603264 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:27.701441 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74e276-d183-4f98-b1ff-424409376f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:27.702407 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74e276-d183-4f98-b1ff-424409376f6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:27.704930 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74e276-d183-4f98-b1ff-424409376f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:27.705441 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74e276-d183-4f98-b1ff-424409376f6a admin admin] PATH_INFO: `/v3/users/40468f6c404f40708a0a48863bdfc316` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:27.707143 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b74e276-d183-4f98-b1ff-424409376f6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=40468f6c404f40708a0a48863bdfc316)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:27.709578 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b74e276-d183-4f98-b1ff-424409376f6a admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:27.960136 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea769eb-d203-4889-be97-9569569ba81d admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:27.961769 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-8ea769eb-d203-4889-be97-9569569ba81d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:27.975019 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-7b74e276-d183-4f98-b1ff-424409376f6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40468f6c404f40708a0a48863bdfc316'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:27.978601 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea769eb-d203-4889-be97-9569569ba81d admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:27.986563 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-7b74e276-d183-4f98-b1ff-424409376f6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40468f6c404f40708a0a48863bdfc316'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:27.988846 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31390918-b3c7-46a2-a05d-d9235dc9e3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:27.990794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31390918-b3c7-46a2-a05d-d9235dc9e3d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:27.999005 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31390918-b3c7-46a2-a05d-d9235dc9e3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:28.004272 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31390918-b3c7-46a2-a05d-d9235dc9e3d7 admin admin] PATH_INFO: `/v3/projects/569a0a23d3564dc2937967da46f1299e` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:28.004272 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31390918-b3c7-46a2-a05d-d9235dc9e3d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=569a0a23d3564dc2937967da46f1299e)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:28.004272 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31390918-b3c7-46a2-a05d-d9235dc9e3d7 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:28.004988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-7b74e276-d183-4f98-b1ff-424409376f6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40468f6c404f40708a0a48863bdfc316'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:28.005304 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-7b74e276-d183-4f98-b1ff-424409376f6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40468f6c404f40708a0a48863bdfc316'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:28.006068 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-7b74e276-d183-4f98-b1ff-424409376f6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40468f6c404f40708a0a48863bdfc316'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:28.072835 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1127/2285] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:29:27 2022] DELETE /identity/v3/users/40468f6c404f40708a0a48863bdfc316 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:29:28.126517 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-31390918-b3c7-46a2-a05d-d9235dc9e3d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '569a0a23d3564dc2937967da46f1299e', 'request_id': 'req-31390918-b3c7-46a2-a05d-d9235dc9e3d7'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:28.133588 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-31390918-b3c7-46a2-a05d-d9235dc9e3d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '569a0a23d3564dc2937967da46f1299e', 'request_id': 'req-31390918-b3c7-46a2-a05d-d9235dc9e3d7'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:28.237587 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-31390918-b3c7-46a2-a05d-d9235dc9e3d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '569a0a23d3564dc2937967da46f1299e', 'request_id': 'req-31390918-b3c7-46a2-a05d-d9235dc9e3d7'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:28.252934 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-31390918-b3c7-46a2-a05d-d9235dc9e3d7 admin admin] The token cache is being invalidate because project 569a0a23d3564dc2937967da46f1299e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:29:28.253196 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-31390918-b3c7-46a2-a05d-d9235dc9e3d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:28.254064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-31390918-b3c7-46a2-a05d-d9235dc9e3d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:28.255677 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1159/2286] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:29:27 2022] DELETE /identity/v3/projects/569a0a23d3564dc2937967da46f1299e => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:29:28.259711 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74e276-d183-4f98-b1ff-424409376f6a admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:28.261262 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7b74e276-d183-4f98-b1ff-424409376f6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:28.336075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74e276-d183-4f98-b1ff-424409376f6a admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:28.385033 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:28.415242 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38732227-239e-40e6-8f79-6875de87cff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:28.416138 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38732227-239e-40e6-8f79-6875de87cff2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:28.416419 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38732227-239e-40e6-8f79-6875de87cff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:28.416724 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38732227-239e-40e6-8f79-6875de87cff2 admin admin] PATH_INFO: `/v3/users/cfa8d8d688bd4560a66df89bd5ec20dd` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:28.417984 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38732227-239e-40e6-8f79-6875de87cff2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfa8d8d688bd4560a66df89bd5ec20dd)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:28.420083 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38732227-239e-40e6-8f79-6875de87cff2 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:28.571519 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31390918-b3c7-46a2-a05d-d9235dc9e3d7 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:28.572690 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-31390918-b3c7-46a2-a05d-d9235dc9e3d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:28.587445 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31390918-b3c7-46a2-a05d-d9235dc9e3d7 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:28.592663 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e492334-e8c0-4fd1-b36a-6bea7dfb4c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:28.594281 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e492334-e8c0-4fd1-b36a-6bea7dfb4c9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:28.600036 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e492334-e8c0-4fd1-b36a-6bea7dfb4c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:28.600036 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e492334-e8c0-4fd1-b36a-6bea7dfb4c9a admin admin] PATH_INFO: `/v3/projects/ae587556ebd1482aa353f60d58bcd89c` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:28.600036 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e492334-e8c0-4fd1-b36a-6bea7dfb4c9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae587556ebd1482aa353f60d58bcd89c)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:28.600036 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e492334-e8c0-4fd1-b36a-6bea7dfb4c9a admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:28.634529 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-38732227-239e-40e6-8f79-6875de87cff2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfa8d8d688bd4560a66df89bd5ec20dd'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:28.642123 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-38732227-239e-40e6-8f79-6875de87cff2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfa8d8d688bd4560a66df89bd5ec20dd'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:28.654077 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-38732227-239e-40e6-8f79-6875de87cff2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfa8d8d688bd4560a66df89bd5ec20dd'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:28.654853 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-38732227-239e-40e6-8f79-6875de87cff2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfa8d8d688bd4560a66df89bd5ec20dd'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:28.659929 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-38732227-239e-40e6-8f79-6875de87cff2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfa8d8d688bd4560a66df89bd5ec20dd'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:28.709327 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:28.712182 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4e492334-e8c0-4fd1-b36a-6bea7dfb4c9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae587556ebd1482aa353f60d58bcd89c', 'request_id': 'req-4e492334-e8c0-4fd1-b36a-6bea7dfb4c9a'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:28.713625 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1128/2287] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:29:28 2022] DELETE /identity/v3/users/cfa8d8d688bd4560a66df89bd5ec20dd => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:29:28.720544 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4e492334-e8c0-4fd1-b36a-6bea7dfb4c9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae587556ebd1482aa353f60d58bcd89c', 'request_id': 'req-4e492334-e8c0-4fd1-b36a-6bea7dfb4c9a'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:28.775190 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4e492334-e8c0-4fd1-b36a-6bea7dfb4c9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae587556ebd1482aa353f60d58bcd89c', 'request_id': 'req-4e492334-e8c0-4fd1-b36a-6bea7dfb4c9a'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:28.797393 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4e492334-e8c0-4fd1-b36a-6bea7dfb4c9a admin admin] The token cache is being invalidate because project ae587556ebd1482aa353f60d58bcd89c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:29:28.801461 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4e492334-e8c0-4fd1-b36a-6bea7dfb4c9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:28.802690 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4e492334-e8c0-4fd1-b36a-6bea7dfb4c9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:28.805474 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1160/2288] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:29:28 2022] DELETE /identity/v3/projects/ae587556ebd1482aa353f60d58bcd89c => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:29:28.809957 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38732227-239e-40e6-8f79-6875de87cff2 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:28.811497 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-38732227-239e-40e6-8f79-6875de87cff2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:28.889140 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38732227-239e-40e6-8f79-6875de87cff2 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:28.894848 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5725f5-c3d1-4c6b-9018-177fb4518975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:28.895989 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5725f5-c3d1-4c6b-9018-177fb4518975 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:28.896837 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5725f5-c3d1-4c6b-9018-177fb4518975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:28.897167 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5725f5-c3d1-4c6b-9018-177fb4518975 admin admin] PATH_INFO: `/v3/users/73432fdcfb0446dcb6e43e2e56a14ca2` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:28.898438 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5725f5-c3d1-4c6b-9018-177fb4518975 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73432fdcfb0446dcb6e43e2e56a14ca2)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:28.900428 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5725f5-c3d1-4c6b-9018-177fb4518975 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:29.139338 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9c5725f5-c3d1-4c6b-9018-177fb4518975 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73432fdcfb0446dcb6e43e2e56a14ca2'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:29.146920 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9c5725f5-c3d1-4c6b-9018-177fb4518975 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73432fdcfb0446dcb6e43e2e56a14ca2'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:29.159070 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9c5725f5-c3d1-4c6b-9018-177fb4518975 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73432fdcfb0446dcb6e43e2e56a14ca2'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:29.160004 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9c5725f5-c3d1-4c6b-9018-177fb4518975 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73432fdcfb0446dcb6e43e2e56a14ca2'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:29.160674 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9c5725f5-c3d1-4c6b-9018-177fb4518975 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73432fdcfb0446dcb6e43e2e56a14ca2'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:29.164525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:29.246732 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e492334-e8c0-4fd1-b36a-6bea7dfb4c9a admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:29.248331 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4e492334-e8c0-4fd1-b36a-6bea7dfb4c9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:29.275478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1129/2289] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:29:28 2022] DELETE /identity/v3/users/73432fdcfb0446dcb6e43e2e56a14ca2 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:29:29.340949 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e492334-e8c0-4fd1-b36a-6bea7dfb4c9a admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:29.404648 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c33487-12f6-46e3-8cf1-2ed40e1dc101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:29.406096 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c33487-12f6-46e3-8cf1-2ed40e1dc101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:29.406501 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c33487-12f6-46e3-8cf1-2ed40e1dc101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:29.406791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c33487-12f6-46e3-8cf1-2ed40e1dc101 admin admin] PATH_INFO: `/v3/projects/2ffa0c4279a84e0085384a8f72c1fdc8` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:29.408004 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c33487-12f6-46e3-8cf1-2ed40e1dc101 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ffa0c4279a84e0085384a8f72c1fdc8)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:29.409978 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c33487-12f6-46e3-8cf1-2ed40e1dc101 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:29.535503 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-c3c33487-12f6-46e3-8cf1-2ed40e1dc101 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ffa0c4279a84e0085384a8f72c1fdc8', 'request_id': 'req-c3c33487-12f6-46e3-8cf1-2ed40e1dc101'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:29.542585 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-c3c33487-12f6-46e3-8cf1-2ed40e1dc101 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ffa0c4279a84e0085384a8f72c1fdc8', 'request_id': 'req-c3c33487-12f6-46e3-8cf1-2ed40e1dc101'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:29.638141 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-c3c33487-12f6-46e3-8cf1-2ed40e1dc101 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ffa0c4279a84e0085384a8f72c1fdc8', 'request_id': 'req-c3c33487-12f6-46e3-8cf1-2ed40e1dc101'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:29.664976 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-c3c33487-12f6-46e3-8cf1-2ed40e1dc101 admin admin] The token cache is being invalidate because project 2ffa0c4279a84e0085384a8f72c1fdc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:29:29.664976 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-c3c33487-12f6-46e3-8cf1-2ed40e1dc101 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:29.664976 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-c3c33487-12f6-46e3-8cf1-2ed40e1dc101 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:29.664976 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1161/2290] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:29:29 2022] DELETE /identity/v3/projects/2ffa0c4279a84e0085384a8f72c1fdc8 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:29:29.666096 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d6ebf-8761-4b03-8355-ca792b01272f None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:29.666563 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d6ebf-8761-4b03-8355-ca792b01272f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:29.666651 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d6ebf-8761-4b03-8355-ca792b01272f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:29.684999 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-707d6ebf-8761-4b03-8355-ca792b01272f None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:29.686234 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-707d6ebf-8761-4b03-8355-ca792b01272f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:29.749239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1130/2291] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:29:29.757914 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08bfb3-2b34-4344-ba50-c27d3faac986 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:29.758295 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08bfb3-2b34-4344-ba50-c27d3faac986 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:29.758863 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08bfb3-2b34-4344-ba50-c27d3faac986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:29.778020 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-dc08bfb3-2b34-4344-ba50-c27d3faac986 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:29.779386 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-dc08bfb3-2b34-4344-ba50-c27d3faac986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:29.799082 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1162/2292] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:29:29.812439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d6ebf-8761-4b03-8355-ca792b01272f None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:29.822482 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d6ebf-8761-4b03-8355-ca792b01272f None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:29.828050 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a9d06-b91c-4369-9ff7-db3642d58efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:29.828947 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a9d06-b91c-4369-9ff7-db3642d58efd admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:29.829241 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a9d06-b91c-4369-9ff7-db3642d58efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:29.829525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a9d06-b91c-4369-9ff7-db3642d58efd admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:29.830272 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a9d06-b91c-4369-9ff7-db3642d58efd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:29:29.830599 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a9d06-b91c-4369-9ff7-db3642d58efd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:29.832225 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a9d06-b91c-4369-9ff7-db3642d58efd admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:29.834988 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc08bfb3-2b34-4344-ba50-c27d3faac986 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:29.836439 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-dc08bfb3-2b34-4344-ba50-c27d3faac986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:29.846882 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1131/2293] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:29:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:29:29.848489 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:29.851553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a9d06-b91c-4369-9ff7-db3642d58efd admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:29.863043 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a9d06-b91c-4369-9ff7-db3642d58efd admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:29.863899 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc08bfb3-2b34-4344-ba50-c27d3faac986 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:29.868556 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63284f8-d136-4b28-86bb-850eee4f3671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:29.869809 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63284f8-d136-4b28-86bb-850eee4f3671 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:29.870102 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63284f8-d136-4b28-86bb-850eee4f3671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:29.870378 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63284f8-d136-4b28-86bb-850eee4f3671 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:29.871111 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63284f8-d136-4b28-86bb-850eee4f3671 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:29.871249 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc126b9-e189-4a02-95e4-d2c654b79cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:29.872194 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc126b9-e189-4a02-95e4-d2c654b79cf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:29.872596 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc126b9-e189-4a02-95e4-d2c654b79cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:29.872942 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc126b9-e189-4a02-95e4-d2c654b79cf1 admin admin] PATH_INFO: `/v3/projects/c50588dbcfd54bb0b28556064c9b2d50` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:29.873561 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63284f8-d136-4b28-86bb-850eee4f3671 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:29.874517 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc126b9-e189-4a02-95e4-d2c654b79cf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c50588dbcfd54bb0b28556064c9b2d50)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:29.876433 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc126b9-e189-4a02-95e4-d2c654b79cf1 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:29.991481 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1132/2294] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:29:29 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:29:29.996123 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63284f8-d136-4b28-86bb-850eee4f3671 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:30.003320 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-dcc126b9-e189-4a02-95e4-d2c654b79cf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c50588dbcfd54bb0b28556064c9b2d50', 'request_id': 'req-dcc126b9-e189-4a02-95e4-d2c654b79cf1'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:30.010238 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-dcc126b9-e189-4a02-95e4-d2c654b79cf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c50588dbcfd54bb0b28556064c9b2d50', 'request_id': 'req-dcc126b9-e189-4a02-95e4-d2c654b79cf1'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:30.067673 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63284f8-d136-4b28-86bb-850eee4f3671 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:30.072396 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdd35d9-31ef-4961-aba0-27c5063a4c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:30.073108 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd35d9-31ef-4961-aba0-27c5063a4c67 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:30.073392 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd35d9-31ef-4961-aba0-27c5063a4c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:30.073740 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd35d9-31ef-4961-aba0-27c5063a4c67 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:30.074498 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdd35d9-31ef-4961-aba0-27c5063a4c67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:30.076479 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdd35d9-31ef-4961-aba0-27c5063a4c67 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:30.100789 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-dcc126b9-e189-4a02-95e4-d2c654b79cf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c50588dbcfd54bb0b28556064c9b2d50', 'request_id': 'req-dcc126b9-e189-4a02-95e4-d2c654b79cf1'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:30.112398 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-dcc126b9-e189-4a02-95e4-d2c654b79cf1 admin admin] The token cache is being invalidate because project c50588dbcfd54bb0b28556064c9b2d50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:29:30.112857 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-dcc126b9-e189-4a02-95e4-d2c654b79cf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:30.113957 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-dcc126b9-e189-4a02-95e4-d2c654b79cf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:30.115533 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1163/2295] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:29:29 2022] DELETE /identity/v3/projects/c50588dbcfd54bb0b28556064c9b2d50 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:29:30.121076 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcffd4d-426d-41eb-b37e-61a112c9e079 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:30.121425 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcffd4d-426d-41eb-b37e-61a112c9e079 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:30.121716 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcffd4d-426d-41eb-b37e-61a112c9e079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:30.140607 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-4fcffd4d-426d-41eb-b37e-61a112c9e079 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:30.141987 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4fcffd4d-426d-41eb-b37e-61a112c9e079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:30.150004 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1164/2296] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:29:30.159529 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6026a3-6717-43c1-b9a2-35c09957805d None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:30.159825 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6026a3-6717-43c1-b9a2-35c09957805d None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:30.160105 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6026a3-6717-43c1-b9a2-35c09957805d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:30.178010 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-9f6026a3-6717-43c1-b9a2-35c09957805d None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:30.179252 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-9f6026a3-6717-43c1-b9a2-35c09957805d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:30.195237 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1165/2297] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:29:30.199938 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6026a3-6717-43c1-b9a2-35c09957805d None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:30.210469 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6026a3-6717-43c1-b9a2-35c09957805d None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:30.215529 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21596ebe-5708-45f2-ae78-bcb7b0dba816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:30.216213 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21596ebe-5708-45f2-ae78-bcb7b0dba816 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:30.216538 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21596ebe-5708-45f2-ae78-bcb7b0dba816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:30.216820 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21596ebe-5708-45f2-ae78-bcb7b0dba816 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:30.217565 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21596ebe-5708-45f2-ae78-bcb7b0dba816 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:29:30.217952 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21596ebe-5708-45f2-ae78-bcb7b0dba816 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:30.219520 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21596ebe-5708-45f2-ae78-bcb7b0dba816 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:30.232434 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1166/2298] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:29:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:29:30.237336 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21596ebe-5708-45f2-ae78-bcb7b0dba816 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:30.246775 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1133/2299] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:29:29 2022] POST /identity/v3/users => generated 293 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:29:30.247993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21596ebe-5708-45f2-ae78-bcb7b0dba816 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:30.251923 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdd35d9-31ef-4961-aba0-27c5063a4c67 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:30.255531 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788a091-43f0-45df-9202-3f3309dbc3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:30.256244 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788a091-43f0-45df-9202-3f3309dbc3d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:30.256490 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-afdd35d9-31ef-4961-aba0-27c5063a4c67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:30.256625 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788a091-43f0-45df-9202-3f3309dbc3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:30.256902 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788a091-43f0-45df-9202-3f3309dbc3d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:30.258294 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1788a091-43f0-45df-9202-3f3309dbc3d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:30.264652 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1788a091-43f0-45df-9202-3f3309dbc3d3 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:30.281725 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdd35d9-31ef-4961-aba0-27c5063a4c67 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:30.286628 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978fb8cd-7b8f-4d21-aae1-c98abf8555f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:30.287315 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fb8cd-7b8f-4d21-aae1-c98abf8555f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:30.287625 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fb8cd-7b8f-4d21-aae1-c98abf8555f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:30.288006 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fb8cd-7b8f-4d21-aae1-c98abf8555f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:30.288799 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978fb8cd-7b8f-4d21-aae1-c98abf8555f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:30.290442 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978fb8cd-7b8f-4d21-aae1-c98abf8555f8 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:30.301146 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1134/2300] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:29:30.305215 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978fb8cd-7b8f-4d21-aae1-c98abf8555f8 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:30.316503 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978fb8cd-7b8f-4d21-aae1-c98abf8555f8 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:30.321091 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2660a9e4-001d-42bd-baab-d6ef5ff4f13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:30.321750 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660a9e4-001d-42bd-baab-d6ef5ff4f13c admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:30.322035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660a9e4-001d-42bd-baab-d6ef5ff4f13c admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:30.322313 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660a9e4-001d-42bd-baab-d6ef5ff4f13c admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:30.323045 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2660a9e4-001d-42bd-baab-d6ef5ff4f13c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:30.324665 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2660a9e4-001d-42bd-baab-d6ef5ff4f13c admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:30.336950 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1135/2301] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 06:29:30.341115 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2660a9e4-001d-42bd-baab-d6ef5ff4f13c admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:30.342055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1167/2302] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:29:30 2022] POST /identity/v3/projects => generated 371 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:29:30.343035 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788a091-43f0-45df-9202-3f3309dbc3d3 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:30.351652 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2660a9e4-001d-42bd-baab-d6ef5ff4f13c admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:30.356484 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:30.411699 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788a091-43f0-45df-9202-3f3309dbc3d3 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:30.415740 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42695d7-e97a-4991-a59c-c6088b419b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:30.416643 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62424d5-5268-4c32-b554-e71c05440055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:30.416774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42695d7-e97a-4991-a59c-c6088b419b21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:30.416937 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42695d7-e97a-4991-a59c-c6088b419b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:30.417218 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42695d7-e97a-4991-a59c-c6088b419b21 admin admin] PATH_INFO: `/v3/projects/7144dbf0b1674af681ec46c5859fdd7b/users/fcbfc2924ebb47f384bd621b874760fe/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:30.417302 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62424d5-5268-4c32-b554-e71c05440055 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:30.417585 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62424d5-5268-4c32-b554-e71c05440055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:30.417869 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62424d5-5268-4c32-b554-e71c05440055 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:30.418560 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62424d5-5268-4c32-b554-e71c05440055 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:30.420694 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62424d5-5268-4c32-b554-e71c05440055 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:30.439912 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42695d7-e97a-4991-a59c-c6088b419b21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7144dbf0b1674af681ec46c5859fdd7b, user_id=fcbfc2924ebb47f384bd621b874760fe, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:30.442655 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42695d7-e97a-4991-a59c-c6088b419b21 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:30.527829 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1168/2304] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:29:30 2022] POST /identity/v3/users => generated 286 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:29:30.529173 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62424d5-5268-4c32-b554-e71c05440055 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:30.529743 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1136/2304] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:29:30 2022] PUT /identity/v3/projects/7144dbf0b1674af681ec46c5859fdd7b/users/fcbfc2924ebb47f384bd621b874760fe/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:29:30.537680 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d0a4e-7b87-4158-ae51-9dcfbeba8b1d None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:30.538199 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d0a4e-7b87-4158-ae51-9dcfbeba8b1d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:30.538697 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d0a4e-7b87-4158-ae51-9dcfbeba8b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:30.544727 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62424d5-5268-4c32-b554-e71c05440055 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:30.576579 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:30.579360 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-9d0d0a4e-7b87-4158-ae51-9dcfbeba8b1d None None] MFA Rules not processed for user `fcbfc2924ebb47f384bd621b874760fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:30.581524 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-9d0d0a4e-7b87-4158-ae51-9dcfbeba8b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:30.608086 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def0bf8e-058b-4660-97e4-22d037b72c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:30.609408 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0bf8e-058b-4660-97e4-22d037b72c81 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:30.609694 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0bf8e-058b-4660-97e4-22d037b72c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:30.609975 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0bf8e-058b-4660-97e4-22d037b72c81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:30.610717 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def0bf8e-058b-4660-97e4-22d037b72c81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:30.612360 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def0bf8e-058b-4660-97e4-22d037b72c81 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:30.623976 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1169/2305] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:29:30.629368 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def0bf8e-058b-4660-97e4-22d037b72c81 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:30.641662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def0bf8e-058b-4660-97e4-22d037b72c81 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:30.646781 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8829933-3890-479f-b096-11544a410e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:30.647568 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8829933-3890-479f-b096-11544a410e32 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:30.647855 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8829933-3890-479f-b096-11544a410e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:30.648134 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8829933-3890-479f-b096-11544a410e32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:30.648920 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8829933-3890-479f-b096-11544a410e32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:30.650568 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8829933-3890-479f-b096-11544a410e32 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:30.665467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1170/2306] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 06:29:30.667359 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8829933-3890-479f-b096-11544a410e32 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:30.682683 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8829933-3890-479f-b096-11544a410e32 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:30.690687 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1137/2307] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:29:30.697339 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d0a4e-7b87-4158-ae51-9dcfbeba8b1d None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:30.698657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5898580-1016-4a5e-9227-0d23e660fc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:30.699967 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5898580-1016-4a5e-9227-0d23e660fc08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:30.701056 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5898580-1016-4a5e-9227-0d23e660fc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:30.701828 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5898580-1016-4a5e-9227-0d23e660fc08 admin admin] PATH_INFO: `/v3/projects/123fd072dc8b4ac095c1ee3579798e4b/users/c75ed30649124403b3f1f2f8bdc6a377/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:30.713040 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d0a4e-7b87-4158-ae51-9dcfbeba8b1d None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:30.718192 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c747aa-b872-4e60-9826-2ae94dc312d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:30.718954 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c747aa-b872-4e60-9826-2ae94dc312d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:30.719258 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c747aa-b872-4e60-9826-2ae94dc312d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:30.719535 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c747aa-b872-4e60-9826-2ae94dc312d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:30.720237 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c747aa-b872-4e60-9826-2ae94dc312d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:30.722435 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c747aa-b872-4e60-9826-2ae94dc312d3 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:30.723277 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5898580-1016-4a5e-9227-0d23e660fc08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=123fd072dc8b4ac095c1ee3579798e4b, user_id=c75ed30649124403b3f1f2f8bdc6a377, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:30.726313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5898580-1016-4a5e-9227-0d23e660fc08 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:30.793600 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1171/2308] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:29:30 2022] PUT /identity/v3/projects/123fd072dc8b4ac095c1ee3579798e4b/users/c75ed30649124403b3f1f2f8bdc6a377/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:29:30.799496 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c47661-4e27-46b3-bf84-dfe88b56badf None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:30.799787 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c47661-4e27-46b3-bf84-dfe88b56badf None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:30.800094 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c47661-4e27-46b3-bf84-dfe88b56badf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:30.820143 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1138/2309] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:29:30 2022] POST /identity/v3/projects => generated 387 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:29:30.824884 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c747aa-b872-4e60-9826-2ae94dc312d3 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:30.825464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-34c47661-4e27-46b3-bf84-dfe88b56badf None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:30.828083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-34c47661-4e27-46b3-bf84-dfe88b56badf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:30.893823 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1172/2310] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:29:30.902635 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c747aa-b872-4e60-9826-2ae94dc312d3 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:30.906126 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c47661-4e27-46b3-bf84-dfe88b56badf None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:30.907614 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-34c47661-4e27-46b3-bf84-dfe88b56badf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:30.908579 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da8bb9-9ffb-4d01-81f0-0170f16fe09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:30.909261 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da8bb9-9ffb-4d01-81f0-0170f16fe09b admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:30.909551 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da8bb9-9ffb-4d01-81f0-0170f16fe09b admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:30.909832 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da8bb9-9ffb-4d01-81f0-0170f16fe09b admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:30.910708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da8bb9-9ffb-4d01-81f0-0170f16fe09b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:30.913227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da8bb9-9ffb-4d01-81f0-0170f16fe09b admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:30.922388 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:30.981590 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c47661-4e27-46b3-bf84-dfe88b56badf None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:30.985840 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2bc2d0-2b20-4b07-a331-a3a03d31b827 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:30.986505 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2bc2d0-2b20-4b07-a331-a3a03d31b827 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:30.986788 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2bc2d0-2b20-4b07-a331-a3a03d31b827 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:30.987076 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2bc2d0-2b20-4b07-a331-a3a03d31b827 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:30.999700 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2bc2d0-2b20-4b07-a331-a3a03d31b827 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:31.001375 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2bc2d0-2b20-4b07-a331-a3a03d31b827 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:31.013720 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1139/2311] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:29:30 2022] POST /identity/v3/users => generated 300 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:29:31.013839 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1173/2312] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:29:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 06:29:31.015625 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da8bb9-9ffb-4d01-81f0-0170f16fe09b admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:31.028580 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da8bb9-9ffb-4d01-81f0-0170f16fe09b admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:31.037101 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954f66c7-3f32-4fe7-a58a-f562e58df186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:31.038098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954f66c7-3f32-4fe7-a58a-f562e58df186 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:31.038401 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954f66c7-3f32-4fe7-a58a-f562e58df186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:31.038670 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954f66c7-3f32-4fe7-a58a-f562e58df186 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:31.039432 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954f66c7-3f32-4fe7-a58a-f562e58df186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:31.041779 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954f66c7-3f32-4fe7-a58a-f562e58df186 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:31.055522 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1140/2313] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:29:31.057319 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2bc2d0-2b20-4b07-a331-a3a03d31b827 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:31.068247 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2bc2d0-2b20-4b07-a331-a3a03d31b827 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:31.072895 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0971bb82-b7f2-42c3-af75-e4b0d6d3bb15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:31.073773 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971bb82-b7f2-42c3-af75-e4b0d6d3bb15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:31.074052 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971bb82-b7f2-42c3-af75-e4b0d6d3bb15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:31.074326 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971bb82-b7f2-42c3-af75-e4b0d6d3bb15 admin admin] PATH_INFO: `/v3/projects/e8bc9ce7ca73453a85f240b8ff45155b/users/bd4139a9b71348578d132237835e9874/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:31.091815 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0971bb82-b7f2-42c3-af75-e4b0d6d3bb15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8bc9ce7ca73453a85f240b8ff45155b, user_id=bd4139a9b71348578d132237835e9874, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:31.095097 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0971bb82-b7f2-42c3-af75-e4b0d6d3bb15 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:31.206723 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1174/2314] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:29:31 2022] PUT /identity/v3/projects/e8bc9ce7ca73453a85f240b8ff45155b/users/bd4139a9b71348578d132237835e9874/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:29:31.211051 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954f66c7-3f32-4fe7-a58a-f562e58df186 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:31.226459 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954f66c7-3f32-4fe7-a58a-f562e58df186 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:31.316919 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8336dc9-5965-4dc2-97f2-7165a2b1174f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:31.317590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8336dc9-5965-4dc2-97f2-7165a2b1174f admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:31.317902 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8336dc9-5965-4dc2-97f2-7165a2b1174f admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:31.318182 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8336dc9-5965-4dc2-97f2-7165a2b1174f admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:31.318916 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8336dc9-5965-4dc2-97f2-7165a2b1174f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:31.320532 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8336dc9-5965-4dc2-97f2-7165a2b1174f admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:31.332004 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1141/2315] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:29:31.336450 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0971bb82-b7f2-42c3-af75-e4b0d6d3bb15 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:31.348065 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0971bb82-b7f2-42c3-af75-e4b0d6d3bb15 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:31.352922 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde8eee2-b3ce-47d4-b140-fca53d5a0fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:31.353896 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8eee2-b3ce-47d4-b140-fca53d5a0fb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:31.354247 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8eee2-b3ce-47d4-b140-fca53d5a0fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:31.354593 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8eee2-b3ce-47d4-b140-fca53d5a0fb9 admin admin] PATH_INFO: `/v3/projects/e8bc9ce7ca73453a85f240b8ff45155b/users/bd4139a9b71348578d132237835e9874/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:31.357026 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde8eee2-b3ce-47d4-b140-fca53d5a0fb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8bc9ce7ca73453a85f240b8ff45155b, user_id=bd4139a9b71348578d132237835e9874, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:31.359829 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde8eee2-b3ce-47d4-b140-fca53d5a0fb9 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:31.363572 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:31.372031 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1175/2316] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:29:31 2022] PUT /identity/v3/projects/e8bc9ce7ca73453a85f240b8ff45155b/users/bd4139a9b71348578d132237835e9874/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:29:31.383075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef05e0d-b405-45a7-a6eb-54ce1a40abca None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:31.383454 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef05e0d-b405-45a7-a6eb-54ce1a40abca None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:31.383796 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef05e0d-b405-45a7-a6eb-54ce1a40abca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:31.420665 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-3ef05e0d-b405-45a7-a6eb-54ce1a40abca None None] MFA Rules not processed for user `bd4139a9b71348578d132237835e9874`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:31.422266 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-3ef05e0d-b405-45a7-a6eb-54ce1a40abca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:31.533386 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1142/2317] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:31 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:29:31.544317 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde8eee2-b3ce-47d4-b140-fca53d5a0fb9 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:31.546368 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-fde8eee2-b3ce-47d4-b140-fca53d5a0fb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:31.631215 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde8eee2-b3ce-47d4-b140-fca53d5a0fb9 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:31.635223 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d2dc5-3608-435a-9461-f3e898490bdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:31.635908 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d2dc5-3608-435a-9461-f3e898490bdb service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:31.636203 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d2dc5-3608-435a-9461-f3e898490bdb service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:31.642582 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d2dc5-3608-435a-9461-f3e898490bdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:31.650058 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3d2dc5-3608-435a-9461-f3e898490bdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:31.651616 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3d2dc5-3608-435a-9461-f3e898490bdb service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:31.663353 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1176/2318] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:29:31 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:29:33.015980 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3836f5-fbbd-4728-803b-96288b64586a None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:33.016481 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3836f5-fbbd-4728-803b-96288b64586a None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:33.016660 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3836f5-fbbd-4728-803b-96288b64586a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:33.081407 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-8c3836f5-fbbd-4728-803b-96288b64586a None None] MFA Rules not processed for user `c75ed30649124403b3f1f2f8bdc6a377`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:33.082769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8c3836f5-fbbd-4728-803b-96288b64586a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:33.152584 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:33.189509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1143/2319] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:29:33.201446 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d2dc5-3608-435a-9461-f3e898490bdb service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:33.204204 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-fe3d2dc5-3608-435a-9461-f3e898490bdb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:33.219542 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d2dc5-3608-435a-9461-f3e898490bdb service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:33.224446 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c1aac7-955e-47c9-834a-79797488486e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:33.225200 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c1aac7-955e-47c9-834a-79797488486e service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:33.225823 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c1aac7-955e-47c9-834a-79797488486e service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:33.226108 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c1aac7-955e-47c9-834a-79797488486e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:33.237236 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c1aac7-955e-47c9-834a-79797488486e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:33.238906 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c1aac7-955e-47c9-834a-79797488486e service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:33.254229 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1177/2320] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:29:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:29:33.485292 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3836f5-fbbd-4728-803b-96288b64586a None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:33.634680 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3836f5-fbbd-4728-803b-96288b64586a None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:33.643342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d78a6-b073-4923-87ed-751470178b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:33.647118 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d78a6-b073-4923-87ed-751470178b97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:33.647118 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d78a6-b073-4923-87ed-751470178b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:33.647118 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d78a6-b073-4923-87ed-751470178b97 admin admin] PATH_INFO: `/v3/users/fcbfc2924ebb47f384bd621b874760fe` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:33.647591 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4d78a6-b073-4923-87ed-751470178b97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcbfc2924ebb47f384bd621b874760fe)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:33.654260 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4d78a6-b073-4923-87ed-751470178b97 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:33.966682 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3f4d78a6-b073-4923-87ed-751470178b97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcbfc2924ebb47f384bd621b874760fe'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:33.977500 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3f4d78a6-b073-4923-87ed-751470178b97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcbfc2924ebb47f384bd621b874760fe'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:33.997779 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3f4d78a6-b073-4923-87ed-751470178b97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcbfc2924ebb47f384bd621b874760fe'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:33.998590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3f4d78a6-b073-4923-87ed-751470178b97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcbfc2924ebb47f384bd621b874760fe'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:33.999699 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3f4d78a6-b073-4923-87ed-751470178b97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcbfc2924ebb47f384bd621b874760fe'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:34.211815 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1144/2321] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:29:33 2022] DELETE /identity/v3/users/fcbfc2924ebb47f384bd621b874760fe => generated 0 bytes in 726 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:29:34.224078 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b17c3-cc93-4356-9e1f-3f4cdcf10051 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:34.224078 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b17c3-cc93-4356-9e1f-3f4cdcf10051 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:34.224078 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b17c3-cc93-4356-9e1f-3f4cdcf10051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:34.242469 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-7b6b17c3-cc93-4356-9e1f-3f4cdcf10051 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:34.243749 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-7b6b17c3-cc93-4356-9e1f-3f4cdcf10051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:34.348691 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1178/2322] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:29:34.370527 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d78a6-b073-4923-87ed-751470178b97 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:34.373014 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-3f4d78a6-b073-4923-87ed-751470178b97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:34.453772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d78a6-b073-4923-87ed-751470178b97 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:34.457742 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a7e3b3-3c64-4c92-a9fd-151111384191 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:34.458449 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7e3b3-3c64-4c92-a9fd-151111384191 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:34.458742 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7e3b3-3c64-4c92-a9fd-151111384191 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:34.459053 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7e3b3-3c64-4c92-a9fd-151111384191 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:34.470176 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a7e3b3-3c64-4c92-a9fd-151111384191 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:34.471624 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a7e3b3-3c64-4c92-a9fd-151111384191 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:34.482314 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1145/2323] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:29:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:29:35.057260 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b17c3-cc93-4356-9e1f-3f4cdcf10051 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:35.065637 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-7b6b17c3-cc93-4356-9e1f-3f4cdcf10051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:35.071299 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b17c3-cc93-4356-9e1f-3f4cdcf10051 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:35.076246 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792271b-5d9c-4e56-a914-b258cbe4db30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:35.077081 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792271b-5d9c-4e56-a914-b258cbe4db30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:35.078004 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792271b-5d9c-4e56-a914-b258cbe4db30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:35.078287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792271b-5d9c-4e56-a914-b258cbe4db30 admin admin] PATH_INFO: `/v3/projects/7144dbf0b1674af681ec46c5859fdd7b` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:35.079515 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8792271b-5d9c-4e56-a914-b258cbe4db30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7144dbf0b1674af681ec46c5859fdd7b)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:35.081533 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8792271b-5d9c-4e56-a914-b258cbe4db30 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:35.212695 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8792271b-5d9c-4e56-a914-b258cbe4db30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7144dbf0b1674af681ec46c5859fdd7b', 'request_id': 'req-8792271b-5d9c-4e56-a914-b258cbe4db30'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:35.219962 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8792271b-5d9c-4e56-a914-b258cbe4db30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7144dbf0b1674af681ec46c5859fdd7b', 'request_id': 'req-8792271b-5d9c-4e56-a914-b258cbe4db30'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:35.312101 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8792271b-5d9c-4e56-a914-b258cbe4db30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7144dbf0b1674af681ec46c5859fdd7b', 'request_id': 'req-8792271b-5d9c-4e56-a914-b258cbe4db30'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:35.323237 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8792271b-5d9c-4e56-a914-b258cbe4db30 admin admin] The token cache is being invalidate because project 7144dbf0b1674af681ec46c5859fdd7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:29:35.323534 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8792271b-5d9c-4e56-a914-b258cbe4db30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:35.324164 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8792271b-5d9c-4e56-a914-b258cbe4db30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:35.325736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1179/2324] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:29:35 2022] DELETE /identity/v3/projects/7144dbf0b1674af681ec46c5859fdd7b => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:29:35.330581 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a7e3b3-3c64-4c92-a9fd-151111384191 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:35.340590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a7e3b3-3c64-4c92-a9fd-151111384191 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:35.350252 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:35.413149 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dfd363-0523-4b75-8813-bf8f64e4eb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:35.413149 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfd363-0523-4b75-8813-bf8f64e4eb95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:35.413149 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfd363-0523-4b75-8813-bf8f64e4eb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:35.413149 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfd363-0523-4b75-8813-bf8f64e4eb95 admin admin] PATH_INFO: `/v3/users/bd4139a9b71348578d132237835e9874` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:35.413482 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dfd363-0523-4b75-8813-bf8f64e4eb95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd4139a9b71348578d132237835e9874)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:35.415987 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dfd363-0523-4b75-8813-bf8f64e4eb95 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:35.634237 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-a0dfd363-0523-4b75-8813-bf8f64e4eb95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd4139a9b71348578d132237835e9874'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:35.646517 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-a0dfd363-0523-4b75-8813-bf8f64e4eb95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd4139a9b71348578d132237835e9874'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:35.661901 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-a0dfd363-0523-4b75-8813-bf8f64e4eb95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd4139a9b71348578d132237835e9874'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:35.665169 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-a0dfd363-0523-4b75-8813-bf8f64e4eb95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd4139a9b71348578d132237835e9874'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:35.665169 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-a0dfd363-0523-4b75-8813-bf8f64e4eb95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd4139a9b71348578d132237835e9874'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:35.750415 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1146/2325] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:29:35 2022] DELETE /identity/v3/users/bd4139a9b71348578d132237835e9874 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:29:36.325639 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792271b-5d9c-4e56-a914-b258cbe4db30 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:36.329920 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-8792271b-5d9c-4e56-a914-b258cbe4db30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:36.347354 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792271b-5d9c-4e56-a914-b258cbe4db30 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:36.352815 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:36.422771 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49286629-aa6a-4a93-ae7c-5790e9e0d7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:36.423836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49286629-aa6a-4a93-ae7c-5790e9e0d7fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:36.424250 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49286629-aa6a-4a93-ae7c-5790e9e0d7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:36.424629 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49286629-aa6a-4a93-ae7c-5790e9e0d7fd admin admin] PATH_INFO: `/v3/projects/e8bc9ce7ca73453a85f240b8ff45155b` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:36.426136 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49286629-aa6a-4a93-ae7c-5790e9e0d7fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8bc9ce7ca73453a85f240b8ff45155b)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:36.428137 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49286629-aa6a-4a93-ae7c-5790e9e0d7fd admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:36.531778 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a19266-49b1-43c6-ad0b-8d4c75b9d3e4 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:36.532362 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a19266-49b1-43c6-ad0b-8d4c75b9d3e4 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:36.544300 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a19266-49b1-43c6-ad0b-8d4c75b9d3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:36.551687 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-94a19266-49b1-43c6-ad0b-8d4c75b9d3e4 None None] MFA Rules not processed for user `c75ed30649124403b3f1f2f8bdc6a377`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:36.552969 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-94a19266-49b1-43c6-ad0b-8d4c75b9d3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:36.612011 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-49286629-aa6a-4a93-ae7c-5790e9e0d7fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8bc9ce7ca73453a85f240b8ff45155b', 'request_id': 'req-49286629-aa6a-4a93-ae7c-5790e9e0d7fd'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:36.616761 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1147/2326] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:29:36.622334 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-49286629-aa6a-4a93-ae7c-5790e9e0d7fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8bc9ce7ca73453a85f240b8ff45155b', 'request_id': 'req-49286629-aa6a-4a93-ae7c-5790e9e0d7fd'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:36.723765 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-49286629-aa6a-4a93-ae7c-5790e9e0d7fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8bc9ce7ca73453a85f240b8ff45155b', 'request_id': 'req-49286629-aa6a-4a93-ae7c-5790e9e0d7fd'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:36.736109 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-49286629-aa6a-4a93-ae7c-5790e9e0d7fd admin admin] The token cache is being invalidate because project e8bc9ce7ca73453a85f240b8ff45155b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:29:36.736400 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-49286629-aa6a-4a93-ae7c-5790e9e0d7fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:36.737118 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-49286629-aa6a-4a93-ae7c-5790e9e0d7fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:36.738759 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1180/2327] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:29:36 2022] DELETE /identity/v3/projects/e8bc9ce7ca73453a85f240b8ff45155b => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:29:36.744219 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70412a7-67b1-468b-936d-fe084cb106aa None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:36.744718 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70412a7-67b1-468b-936d-fe084cb106aa None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:36.745394 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70412a7-67b1-468b-936d-fe084cb106aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:36.770537 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-c70412a7-67b1-468b-936d-fe084cb106aa None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:36.771782 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-c70412a7-67b1-468b-936d-fe084cb106aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:36.853404 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1148/2328] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:29:36.862002 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e842f-8f67-40d7-be0c-3f0f656fd900 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:36.862346 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e842f-8f67-40d7-be0c-3f0f656fd900 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:36.862623 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e842f-8f67-40d7-be0c-3f0f656fd900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:36.883747 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-334e842f-8f67-40d7-be0c-3f0f656fd900 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:36.890046 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-334e842f-8f67-40d7-be0c-3f0f656fd900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:36.894528 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1181/2329] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:29:36.898789 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70412a7-67b1-468b-936d-fe084cb106aa None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:36.910321 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70412a7-67b1-468b-936d-fe084cb106aa None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:36.919554 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a164ea0-6d79-44e7-a4b5-0f4525a28154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:36.920244 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a164ea0-6d79-44e7-a4b5-0f4525a28154 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:36.921761 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a164ea0-6d79-44e7-a4b5-0f4525a28154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:36.921761 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a164ea0-6d79-44e7-a4b5-0f4525a28154 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:36.922559 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a164ea0-6d79-44e7-a4b5-0f4525a28154 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:29:36.922869 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a164ea0-6d79-44e7-a4b5-0f4525a28154 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:36.924526 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a164ea0-6d79-44e7-a4b5-0f4525a28154 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:36.942732 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1149/2330] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:29:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:29:36.944531 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e842f-8f67-40d7-be0c-3f0f656fd900 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:36.954572 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:36.956440 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e842f-8f67-40d7-be0c-3f0f656fd900 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:36.963965 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b43c8d-c4b2-4e30-86ac-995da9a22dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:36.963965 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b43c8d-c4b2-4e30-86ac-995da9a22dff admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:36.963965 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b43c8d-c4b2-4e30-86ac-995da9a22dff admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:36.963965 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b43c8d-c4b2-4e30-86ac-995da9a22dff admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:36.964459 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b43c8d-c4b2-4e30-86ac-995da9a22dff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:36.964923 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b43c8d-c4b2-4e30-86ac-995da9a22dff admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:37.153652 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1182/2331] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:29:36 2022] POST /identity/v3/projects => generated 373 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:29:37.164659 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a164ea0-6d79-44e7-a4b5-0f4525a28154 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:37.176532 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a164ea0-6d79-44e7-a4b5-0f4525a28154 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:37.243134 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ce45b-bc02-4eb8-9153-90ebea5c5bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:37.243816 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ce45b-bc02-4eb8-9153-90ebea5c5bef admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:37.244355 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ce45b-bc02-4eb8-9153-90ebea5c5bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:37.244681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ce45b-bc02-4eb8-9153-90ebea5c5bef admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:37.245522 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0ce45b-bc02-4eb8-9153-90ebea5c5bef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:37.247456 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0ce45b-bc02-4eb8-9153-90ebea5c5bef admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:37.422953 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1150/2332] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:29:37 2022] POST /identity/v3/users => generated 287 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:29:37.427084 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b43c8d-c4b2-4e30-86ac-995da9a22dff admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:37.451633 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b43c8d-c4b2-4e30-86ac-995da9a22dff admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:37.457751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca32363-c520-4e09-befd-b684af41b382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:37.458908 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca32363-c520-4e09-befd-b684af41b382 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:37.458908 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca32363-c520-4e09-befd-b684af41b382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:37.459475 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca32363-c520-4e09-befd-b684af41b382 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:37.460307 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca32363-c520-4e09-befd-b684af41b382 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:37.462175 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca32363-c520-4e09-befd-b684af41b382 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:37.478409 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1183/2333] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 06:29:37.482464 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ce45b-bc02-4eb8-9153-90ebea5c5bef admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:37.494660 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ce45b-bc02-4eb8-9153-90ebea5c5bef admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:37.500472 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8debcb36-1f48-4ea8-8a0b-a9882593ea02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:37.505849 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8debcb36-1f48-4ea8-8a0b-a9882593ea02 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:37.505849 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8debcb36-1f48-4ea8-8a0b-a9882593ea02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:37.505849 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8debcb36-1f48-4ea8-8a0b-a9882593ea02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:37.505849 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8debcb36-1f48-4ea8-8a0b-a9882593ea02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:37.505849 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8debcb36-1f48-4ea8-8a0b-a9882593ea02 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:37.518759 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1151/2334] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:29:37.523271 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca32363-c520-4e09-befd-b684af41b382 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:37.541545 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca32363-c520-4e09-befd-b684af41b382 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:37.546587 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b9785-2845-4f82-95d7-1808906ff52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:37.547516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b9785-2845-4f82-95d7-1808906ff52b admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:37.547886 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b9785-2845-4f82-95d7-1808906ff52b admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:37.548194 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b9785-2845-4f82-95d7-1808906ff52b admin admin] PATH_INFO: `/v3/projects/74094a5e18014db18674cd6b5da83009/users/0125015df2674ba98a85b0ffc07b5faa/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:37.565150 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b9785-2845-4f82-95d7-1808906ff52b admin admin] RBAC: Authorizing `identity:create_grant(project_id=74094a5e18014db18674cd6b5da83009, user_id=0125015df2674ba98a85b0ffc07b5faa, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:37.567807 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b9785-2845-4f82-95d7-1808906ff52b admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:37.713429 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1184/2335] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:29:37 2022] PUT /identity/v3/projects/74094a5e18014db18674cd6b5da83009/users/0125015df2674ba98a85b0ffc07b5faa/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:29:37.725167 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01dd8e-1afc-4579-baac-71607764b05c None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:37.725609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01dd8e-1afc-4579-baac-71607764b05c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:37.725841 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01dd8e-1afc-4579-baac-71607764b05c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:37.755727 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-ef01dd8e-1afc-4579-baac-71607764b05c None None] MFA Rules not processed for user `0125015df2674ba98a85b0ffc07b5faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:37.756992 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ef01dd8e-1afc-4579-baac-71607764b05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:37.830769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1152/2336] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:29:37.835472 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b9785-2845-4f82-95d7-1808906ff52b admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:37.883610 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:37.910418 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b9785-2845-4f82-95d7-1808906ff52b admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:37.915394 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b0e63e-2619-4189-8d25-1463f3913a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:37.922042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b0e63e-2619-4189-8d25-1463f3913a5d admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:37.922042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b0e63e-2619-4189-8d25-1463f3913a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:37.922042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b0e63e-2619-4189-8d25-1463f3913a5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:37.922042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b0e63e-2619-4189-8d25-1463f3913a5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:37.922042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b0e63e-2619-4189-8d25-1463f3913a5d admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:38.052516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1185/2337] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:29:37 2022] POST /identity/v3/projects => generated 375 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:29:38.056934 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01dd8e-1afc-4579-baac-71607764b05c None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:38.068602 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01dd8e-1afc-4579-baac-71607764b05c None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:38.128027 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115e37e4-ea02-418d-8176-f0061da3862b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:38.128738 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e37e4-ea02-418d-8176-f0061da3862b admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:38.129031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e37e4-ea02-418d-8176-f0061da3862b admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:38.129320 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e37e4-ea02-418d-8176-f0061da3862b admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:38.130034 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115e37e4-ea02-418d-8176-f0061da3862b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:38.132042 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115e37e4-ea02-418d-8176-f0061da3862b admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:38.176540 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b0e63e-2619-4189-8d25-1463f3913a5d admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:38.178443 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d4b0e63e-2619-4189-8d25-1463f3913a5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:38.192807 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b0e63e-2619-4189-8d25-1463f3913a5d admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:38.197546 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea1cc65-cfd2-4c43-93b2-96b53ef999e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:38.198421 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1cc65-cfd2-4c43-93b2-96b53ef999e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:38.198707 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1cc65-cfd2-4c43-93b2-96b53ef999e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:38.199015 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1cc65-cfd2-4c43-93b2-96b53ef999e9 admin admin] PATH_INFO: `/v3/users/bbefc8439c9340cbb30262891a0ce66b` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:38.200187 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea1cc65-cfd2-4c43-93b2-96b53ef999e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbefc8439c9340cbb30262891a0ce66b)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:38.202251 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea1cc65-cfd2-4c43-93b2-96b53ef999e9 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:38.267662 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1153/2338] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:29:38 2022] POST /identity/v3/users => generated 294 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:29:38.271753 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115e37e4-ea02-418d-8176-f0061da3862b admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:38.282954 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115e37e4-ea02-418d-8176-f0061da3862b admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:38.287796 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1205bc54-d0ea-4aba-b0db-edc25642dd4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:38.288468 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1205bc54-d0ea-4aba-b0db-edc25642dd4c admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:38.288924 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1205bc54-d0ea-4aba-b0db-edc25642dd4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:38.289210 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1205bc54-d0ea-4aba-b0db-edc25642dd4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:38.289941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1205bc54-d0ea-4aba-b0db-edc25642dd4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:38.291532 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1205bc54-d0ea-4aba-b0db-edc25642dd4c admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:38.304090 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1154/2339] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 06:29:38.308706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1205bc54-d0ea-4aba-b0db-edc25642dd4c admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:38.319671 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1205bc54-d0ea-4aba-b0db-edc25642dd4c admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:38.324326 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0abd822-b2f2-4a65-b4ab-90c62dc393cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:38.325235 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abd822-b2f2-4a65-b4ab-90c62dc393cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:38.325527 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abd822-b2f2-4a65-b4ab-90c62dc393cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:38.325810 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abd822-b2f2-4a65-b4ab-90c62dc393cb admin admin] PATH_INFO: `/v3/projects/a82def8786e0435ba60a8f95dd260236/users/68658e8f61ff499eb3d66548cae3526f/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:38.342458 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0abd822-b2f2-4a65-b4ab-90c62dc393cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a82def8786e0435ba60a8f95dd260236, user_id=68658e8f61ff499eb3d66548cae3526f, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:38.346719 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0abd822-b2f2-4a65-b4ab-90c62dc393cb admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:38.398113 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:38.450824 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1155/2340] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:29:38 2022] PUT /identity/v3/projects/a82def8786e0435ba60a8f95dd260236/users/68658e8f61ff499eb3d66548cae3526f/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:29:38.454672 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0abd822-b2f2-4a65-b4ab-90c62dc393cb admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:38.458594 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-fea1cc65-cfd2-4c43-93b2-96b53ef999e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbefc8439c9340cbb30262891a0ce66b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:38.470656 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-fea1cc65-cfd2-4c43-93b2-96b53ef999e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbefc8439c9340cbb30262891a0ce66b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:38.471396 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-fea1cc65-cfd2-4c43-93b2-96b53ef999e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbefc8439c9340cbb30262891a0ce66b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:38.472120 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-fea1cc65-cfd2-4c43-93b2-96b53ef999e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbefc8439c9340cbb30262891a0ce66b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:38.495353 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-fea1cc65-cfd2-4c43-93b2-96b53ef999e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbefc8439c9340cbb30262891a0ce66b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:38.535292 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0abd822-b2f2-4a65-b4ab-90c62dc393cb admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:38.549111 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0361ec1b-0a3a-49bd-bc1f-aa4fd68da4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:38.549916 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0361ec1b-0a3a-49bd-bc1f-aa4fd68da4b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:38.551716 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0361ec1b-0a3a-49bd-bc1f-aa4fd68da4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:38.551716 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0361ec1b-0a3a-49bd-bc1f-aa4fd68da4b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:38.551716 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0361ec1b-0a3a-49bd-bc1f-aa4fd68da4b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:38.553200 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0361ec1b-0a3a-49bd-bc1f-aa4fd68da4b4 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:38.569858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1156/2341] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:29:38.571813 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0361ec1b-0a3a-49bd-bc1f-aa4fd68da4b4 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:38.574010 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-0361ec1b-0a3a-49bd-bc1f-aa4fd68da4b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:38.579129 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1186/2342] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:29:38 2022] DELETE /identity/v3/users/bbefc8439c9340cbb30262891a0ce66b => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:29:38.583716 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129189f1-b150-4d18-bcd5-83f08cad55f0 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:38.584087 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129189f1-b150-4d18-bcd5-83f08cad55f0 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:38.584354 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129189f1-b150-4d18-bcd5-83f08cad55f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:38.611095 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0361ec1b-0a3a-49bd-bc1f-aa4fd68da4b4 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:38.618831 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-129189f1-b150-4d18-bcd5-83f08cad55f0 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:38.620329 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-129189f1-b150-4d18-bcd5-83f08cad55f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:38.691529 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef990d3f-8168-43bd-85c1-8a9749c4d5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:38.692489 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef990d3f-8168-43bd-85c1-8a9749c4d5ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:38.692750 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef990d3f-8168-43bd-85c1-8a9749c4d5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:38.693021 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef990d3f-8168-43bd-85c1-8a9749c4d5ca admin admin] PATH_INFO: `/v3/projects/a82def8786e0435ba60a8f95dd260236/users/68658e8f61ff499eb3d66548cae3526f/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:38.695050 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef990d3f-8168-43bd-85c1-8a9749c4d5ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=a82def8786e0435ba60a8f95dd260236, user_id=68658e8f61ff499eb3d66548cae3526f, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:38.697794 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef990d3f-8168-43bd-85c1-8a9749c4d5ca admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:38.707415 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1187/2343] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:29:38.709214 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1157/2344] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:29:38 2022] PUT /identity/v3/projects/a82def8786e0435ba60a8f95dd260236/users/68658e8f61ff499eb3d66548cae3526f/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:29:38.715489 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129189f1-b150-4d18-bcd5-83f08cad55f0 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:38.717101 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-129189f1-b150-4d18-bcd5-83f08cad55f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:38.720482 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d4882-cad8-427d-8e69-1cd391c55b08 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:38.720775 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d4882-cad8-427d-8e69-1cd391c55b08 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:38.721064 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d4882-cad8-427d-8e69-1cd391c55b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:38.758528 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-f16d4882-cad8-427d-8e69-1cd391c55b08 None None] MFA Rules not processed for user `68658e8f61ff499eb3d66548cae3526f`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:38.760787 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f16d4882-cad8-427d-8e69-1cd391c55b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:38.762158 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:38.768902 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:38.803415 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129189f1-b150-4d18-bcd5-83f08cad55f0 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:38.854562 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1158/2345] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:38 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:29:38.865353 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f4f482-c3f7-48a1-9dc4-d98461bcc339 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:38.866005 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4f482-c3f7-48a1-9dc4-d98461bcc339 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:38.866545 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4f482-c3f7-48a1-9dc4-d98461bcc339 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:38.866545 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4f482-c3f7-48a1-9dc4-d98461bcc339 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:38.868287 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d4882-cad8-427d-8e69-1cd391c55b08 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:38.870334 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f16d4882-cad8-427d-8e69-1cd391c55b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:38.944464 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d4882-cad8-427d-8e69-1cd391c55b08 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:38.948323 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30493a3-41db-43b4-8d3d-5bcc78f0af83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:38.949030 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30493a3-41db-43b4-8d3d-5bcc78f0af83 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:38.949309 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30493a3-41db-43b4-8d3d-5bcc78f0af83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:38.949584 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30493a3-41db-43b4-8d3d-5bcc78f0af83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:38.951456 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f30493a3-41db-43b4-8d3d-5bcc78f0af83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:38.965701 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f4f482-c3f7-48a1-9dc4-d98461bcc339 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:38.967222 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f4f482-c3f7-48a1-9dc4-d98461bcc339 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:38.981616 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1188/2346] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:29:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 06:29:39.035470 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30493a3-41db-43b4-8d3d-5bcc78f0af83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:39.037231 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30493a3-41db-43b4-8d3d-5bcc78f0af83 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:39.049673 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1159/2347] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:29:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 06:29:39.582732 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f4f482-c3f7-48a1-9dc4-d98461bcc339 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:39.584251 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-41f4f482-c3f7-48a1-9dc4-d98461bcc339 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:39.600061 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f4f482-c3f7-48a1-9dc4-d98461bcc339 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:39.608008 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e443b6da-5cd6-4161-825e-e288fcd23af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:39.609714 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443b6da-5cd6-4161-825e-e288fcd23af3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:39.610168 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443b6da-5cd6-4161-825e-e288fcd23af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:39.610585 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443b6da-5cd6-4161-825e-e288fcd23af3 admin admin] PATH_INFO: `/v3/projects/3669fc8553d94dd589e027a5d772c8c7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:39.612765 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e443b6da-5cd6-4161-825e-e288fcd23af3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3669fc8553d94dd589e027a5d772c8c7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:39.617112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e443b6da-5cd6-4161-825e-e288fcd23af3 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:39.793676 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e443b6da-5cd6-4161-825e-e288fcd23af3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3669fc8553d94dd589e027a5d772c8c7', 'request_id': 'req-e443b6da-5cd6-4161-825e-e288fcd23af3'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:39.803990 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e443b6da-5cd6-4161-825e-e288fcd23af3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3669fc8553d94dd589e027a5d772c8c7', 'request_id': 'req-e443b6da-5cd6-4161-825e-e288fcd23af3'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:39.924686 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e443b6da-5cd6-4161-825e-e288fcd23af3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3669fc8553d94dd589e027a5d772c8c7', 'request_id': 'req-e443b6da-5cd6-4161-825e-e288fcd23af3'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:39.938476 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e443b6da-5cd6-4161-825e-e288fcd23af3 admin admin] The token cache is being invalidate because project 3669fc8553d94dd589e027a5d772c8c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:29:39.938476 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e443b6da-5cd6-4161-825e-e288fcd23af3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:39.940190 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e443b6da-5cd6-4161-825e-e288fcd23af3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:39.941829 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1189/2348] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:29:39 2022] DELETE /identity/v3/projects/3669fc8553d94dd589e027a5d772c8c7 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:29:43.482886 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30493a3-41db-43b4-8d3d-5bcc78f0af83 service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:43.495272 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30493a3-41db-43b4-8d3d-5bcc78f0af83 service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:43.556946 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:43.568996 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b6897-2f87-4a39-81e9-be4204a77db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:43.570538 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b6897-2f87-4a39-81e9-be4204a77db0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:43.571057 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b6897-2f87-4a39-81e9-be4204a77db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:43.571568 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b6897-2f87-4a39-81e9-be4204a77db0 admin admin] PATH_INFO: `/v3/users/0125015df2674ba98a85b0ffc07b5faa` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:43.573966 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71b6897-2f87-4a39-81e9-be4204a77db0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0125015df2674ba98a85b0ffc07b5faa)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:43.578165 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71b6897-2f87-4a39-81e9-be4204a77db0 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:43.820719 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e71b6897-2f87-4a39-81e9-be4204a77db0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0125015df2674ba98a85b0ffc07b5faa'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:43.830054 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e71b6897-2f87-4a39-81e9-be4204a77db0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0125015df2674ba98a85b0ffc07b5faa'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:43.845803 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e71b6897-2f87-4a39-81e9-be4204a77db0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0125015df2674ba98a85b0ffc07b5faa'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:43.846555 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e71b6897-2f87-4a39-81e9-be4204a77db0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0125015df2674ba98a85b0ffc07b5faa'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:43.847166 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e71b6897-2f87-4a39-81e9-be4204a77db0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0125015df2674ba98a85b0ffc07b5faa'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:43.919418 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1160/2349] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:29:43 2022] DELETE /identity/v3/users/0125015df2674ba98a85b0ffc07b5faa => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:29:43.924365 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d951ed5-9637-41be-b9aa-a96482ee2e47 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:43.925130 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d951ed5-9637-41be-b9aa-a96482ee2e47 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:43.925404 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d951ed5-9637-41be-b9aa-a96482ee2e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:43.946522 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-0d951ed5-9637-41be-b9aa-a96482ee2e47 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:43.947786 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-0d951ed5-9637-41be-b9aa-a96482ee2e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:43.969713 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:44.021971 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1190/2350] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:29:44.032706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b6897-2f87-4a39-81e9-be4204a77db0 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:44.037932 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e71b6897-2f87-4a39-81e9-be4204a77db0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:44.099647 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b6897-2f87-4a39-81e9-be4204a77db0 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:44.103407 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55dc0c7-cf66-4044-8f36-1eba696f841a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:44.104074 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55dc0c7-cf66-4044-8f36-1eba696f841a service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:44.104364 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55dc0c7-cf66-4044-8f36-1eba696f841a service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:44.104679 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55dc0c7-cf66-4044-8f36-1eba696f841a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:44.115577 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55dc0c7-cf66-4044-8f36-1eba696f841a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:44.117087 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55dc0c7-cf66-4044-8f36-1eba696f841a service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:44.129704 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1161/2351] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:29:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 06:29:44.652769 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d951ed5-9637-41be-b9aa-a96482ee2e47 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:44.655780 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-0d951ed5-9637-41be-b9aa-a96482ee2e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:44.677579 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d951ed5-9637-41be-b9aa-a96482ee2e47 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:44.687588 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8879ae-fcde-4e74-83f7-b8914864e3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:44.688797 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8879ae-fcde-4e74-83f7-b8914864e3b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:44.689047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8879ae-fcde-4e74-83f7-b8914864e3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:44.689722 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8879ae-fcde-4e74-83f7-b8914864e3b4 admin admin] PATH_INFO: `/v3/projects/74094a5e18014db18674cd6b5da83009` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:44.690693 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8879ae-fcde-4e74-83f7-b8914864e3b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74094a5e18014db18674cd6b5da83009)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:44.694959 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8879ae-fcde-4e74-83f7-b8914864e3b4 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:44.818449 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6c8879ae-fcde-4e74-83f7-b8914864e3b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74094a5e18014db18674cd6b5da83009', 'request_id': 'req-6c8879ae-fcde-4e74-83f7-b8914864e3b4'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:44.827373 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6c8879ae-fcde-4e74-83f7-b8914864e3b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74094a5e18014db18674cd6b5da83009', 'request_id': 'req-6c8879ae-fcde-4e74-83f7-b8914864e3b4'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:44.889746 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6c8879ae-fcde-4e74-83f7-b8914864e3b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74094a5e18014db18674cd6b5da83009', 'request_id': 'req-6c8879ae-fcde-4e74-83f7-b8914864e3b4'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:44.902687 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6c8879ae-fcde-4e74-83f7-b8914864e3b4 admin admin] The token cache is being invalidate because project 74094a5e18014db18674cd6b5da83009 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:29:44.903008 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6c8879ae-fcde-4e74-83f7-b8914864e3b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:44.903708 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6c8879ae-fcde-4e74-83f7-b8914864e3b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:44.905822 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1191/2352] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:29:44 2022] DELETE /identity/v3/projects/74094a5e18014db18674cd6b5da83009 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:29:44.910203 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55dc0c7-cf66-4044-8f36-1eba696f841a service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:44.920777 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55dc0c7-cf66-4044-8f36-1eba696f841a service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:44.991065 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba3e190-8561-45a1-a46d-4b14960a4a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:44.991956 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3e190-8561-45a1-a46d-4b14960a4a69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:44.992243 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3e190-8561-45a1-a46d-4b14960a4a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:44.992554 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3e190-8561-45a1-a46d-4b14960a4a69 admin admin] PATH_INFO: `/v3/users/68658e8f61ff499eb3d66548cae3526f` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:44.993771 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba3e190-8561-45a1-a46d-4b14960a4a69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68658e8f61ff499eb3d66548cae3526f)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:44.995952 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba3e190-8561-45a1-a46d-4b14960a4a69 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:45.021853 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:45.150801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3ba3e190-8561-45a1-a46d-4b14960a4a69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68658e8f61ff499eb3d66548cae3526f'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:45.159989 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3ba3e190-8561-45a1-a46d-4b14960a4a69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68658e8f61ff499eb3d66548cae3526f'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:45.175234 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3ba3e190-8561-45a1-a46d-4b14960a4a69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68658e8f61ff499eb3d66548cae3526f'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:45.175974 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3ba3e190-8561-45a1-a46d-4b14960a4a69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68658e8f61ff499eb3d66548cae3526f'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:45.176799 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3ba3e190-8561-45a1-a46d-4b14960a4a69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68658e8f61ff499eb3d66548cae3526f'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:45.261540 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1162/2353] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:29:44 2022] DELETE /identity/v3/users/68658e8f61ff499eb3d66548cae3526f => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:29:45.748863 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8879ae-fcde-4e74-83f7-b8914864e3b4 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:45.750730 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-6c8879ae-fcde-4e74-83f7-b8914864e3b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:45.766609 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8879ae-fcde-4e74-83f7-b8914864e3b4 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:45.829472 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4065cebd-b044-4507-a78a-4e5406464ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:45.830287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065cebd-b044-4507-a78a-4e5406464ccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:45.830571 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065cebd-b044-4507-a78a-4e5406464ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:45.830848 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065cebd-b044-4507-a78a-4e5406464ccc admin admin] PATH_INFO: `/v3/projects/a82def8786e0435ba60a8f95dd260236` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:45.832218 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4065cebd-b044-4507-a78a-4e5406464ccc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a82def8786e0435ba60a8f95dd260236)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:45.834233 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4065cebd-b044-4507-a78a-4e5406464ccc admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:45.962047 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:45.963663 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4065cebd-b044-4507-a78a-4e5406464ccc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a82def8786e0435ba60a8f95dd260236', 'request_id': 'req-4065cebd-b044-4507-a78a-4e5406464ccc'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:45.972374 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4065cebd-b044-4507-a78a-4e5406464ccc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a82def8786e0435ba60a8f95dd260236', 'request_id': 'req-4065cebd-b044-4507-a78a-4e5406464ccc'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:46.100832 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4065cebd-b044-4507-a78a-4e5406464ccc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a82def8786e0435ba60a8f95dd260236', 'request_id': 'req-4065cebd-b044-4507-a78a-4e5406464ccc'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:46.115472 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4065cebd-b044-4507-a78a-4e5406464ccc admin admin] The token cache is being invalidate because project a82def8786e0435ba60a8f95dd260236 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:29:46.115840 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4065cebd-b044-4507-a78a-4e5406464ccc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:46.116866 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4065cebd-b044-4507-a78a-4e5406464ccc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:29:46.119695 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1192/2354] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:29:45 2022] DELETE /identity/v3/projects/a82def8786e0435ba60a8f95dd260236 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:29:46.125489 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ccbd3-c3d2-41e4-82e7-fe9d40de8876 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:46.126015 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ccbd3-c3d2-41e4-82e7-fe9d40de8876 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:46.126234 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ccbd3-c3d2-41e4-82e7-fe9d40de8876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:46.152982 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-e69ccbd3-c3d2-41e4-82e7-fe9d40de8876 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:46.154383 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e69ccbd3-c3d2-41e4-82e7-fe9d40de8876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:46.220349 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1163/2355] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:29:46.228385 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ba0fa-1e67-4e5b-b302-e545c7be3df6 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:46.228726 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ba0fa-1e67-4e5b-b302-e545c7be3df6 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:46.229016 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ba0fa-1e67-4e5b-b302-e545c7be3df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:46.248589 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-b57ba0fa-1e67-4e5b-b302-e545c7be3df6 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:46.250471 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-b57ba0fa-1e67-4e5b-b302-e545c7be3df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:46.258785 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1193/2356] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:29:46.263501 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ccbd3-c3d2-41e4-82e7-fe9d40de8876 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:46.275408 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ccbd3-c3d2-41e4-82e7-fe9d40de8876 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:46.279995 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae2a939-44f5-4195-a1b5-411b3878c456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:46.281314 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2a939-44f5-4195-a1b5-411b3878c456 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:46.281602 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2a939-44f5-4195-a1b5-411b3878c456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:46.281886 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2a939-44f5-4195-a1b5-411b3878c456 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:46.282651 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae2a939-44f5-4195-a1b5-411b3878c456 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:29:46.282965 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae2a939-44f5-4195-a1b5-411b3878c456 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:46.284617 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae2a939-44f5-4195-a1b5-411b3878c456 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:46.299002 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1164/2357] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:29:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:29:46.304308 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ba0fa-1e67-4e5b-b302-e545c7be3df6 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:46.314499 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ba0fa-1e67-4e5b-b302-e545c7be3df6 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:46.319230 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194928b2-e7b5-49e7-ab2c-074006d12004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:46.319898 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194928b2-e7b5-49e7-ab2c-074006d12004 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:46.320188 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194928b2-e7b5-49e7-ab2c-074006d12004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:46.320499 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194928b2-e7b5-49e7-ab2c-074006d12004 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:46.321220 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194928b2-e7b5-49e7-ab2c-074006d12004 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:46.323167 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194928b2-e7b5-49e7-ab2c-074006d12004 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:46.473889 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1194/2358] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:29:46 2022] POST /identity/v3/projects => generated 375 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:29:46.478178 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae2a939-44f5-4195-a1b5-411b3878c456 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:46.490899 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae2a939-44f5-4195-a1b5-411b3878c456 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:46.592030 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef714537-bf4c-4b18-8c28-6137f5f6599f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:46.593179 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef714537-bf4c-4b18-8c28-6137f5f6599f admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:46.593649 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef714537-bf4c-4b18-8c28-6137f5f6599f admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:46.594084 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef714537-bf4c-4b18-8c28-6137f5f6599f admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:46.595218 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef714537-bf4c-4b18-8c28-6137f5f6599f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:46.598660 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef714537-bf4c-4b18-8c28-6137f5f6599f admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:46.692318 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1165/2359] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:29:46 2022] POST /identity/v3/users => generated 288 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:29:46.697188 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194928b2-e7b5-49e7-ab2c-074006d12004 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:46.707971 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194928b2-e7b5-49e7-ab2c-074006d12004 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:46.712645 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0319641b-1115-490f-ad55-de6dbcb57b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:46.718953 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319641b-1115-490f-ad55-de6dbcb57b6c admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:46.718953 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319641b-1115-490f-ad55-de6dbcb57b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:46.718953 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319641b-1115-490f-ad55-de6dbcb57b6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:46.718953 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0319641b-1115-490f-ad55-de6dbcb57b6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:46.718953 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0319641b-1115-490f-ad55-de6dbcb57b6c admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:46.728985 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1195/2360] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 06:29:46.734296 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef714537-bf4c-4b18-8c28-6137f5f6599f admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:46.745289 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef714537-bf4c-4b18-8c28-6137f5f6599f admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:46.750384 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f101b6-e58f-4889-ac35-3fcbb1b2cffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:46.751062 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f101b6-e58f-4889-ac35-3fcbb1b2cffe admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:46.751345 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f101b6-e58f-4889-ac35-3fcbb1b2cffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:46.751652 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f101b6-e58f-4889-ac35-3fcbb1b2cffe admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:46.752396 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f101b6-e58f-4889-ac35-3fcbb1b2cffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:46.754023 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f101b6-e58f-4889-ac35-3fcbb1b2cffe admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:46.764954 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1166/2361] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:29:46.769899 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0319641b-1115-490f-ad55-de6dbcb57b6c admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:46.781054 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0319641b-1115-490f-ad55-de6dbcb57b6c admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:46.786026 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0879e-4cb3-49ac-bfda-769103fd9984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:46.786932 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0879e-4cb3-49ac-bfda-769103fd9984 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:46.787219 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0879e-4cb3-49ac-bfda-769103fd9984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:46.787508 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0879e-4cb3-49ac-bfda-769103fd9984 admin admin] PATH_INFO: `/v3/projects/47260a387394403082e8d925797c6285/users/f7c2c5faf28b43b3b879a267b807c006/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:46.804743 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0879e-4cb3-49ac-bfda-769103fd9984 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47260a387394403082e8d925797c6285, user_id=f7c2c5faf28b43b3b879a267b807c006, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:46.807458 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0879e-4cb3-49ac-bfda-769103fd9984 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:46.919048 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1196/2362] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:29:46 2022] PUT /identity/v3/projects/47260a387394403082e8d925797c6285/users/f7c2c5faf28b43b3b879a267b807c006/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:29:46.930936 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca2a45-c4f3-4f6a-994a-34771017ac7d None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:46.931237 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca2a45-c4f3-4f6a-994a-34771017ac7d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:46.931522 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca2a45-c4f3-4f6a-994a-34771017ac7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:46.938126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:46.963352 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-d2ca2a45-c4f3-4f6a-994a-34771017ac7d None None] MFA Rules not processed for user `f7c2c5faf28b43b3b879a267b807c006`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:29:46.964585 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-d2ca2a45-c4f3-4f6a-994a-34771017ac7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:47.041708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1167/2363] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:29:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:29:47.053532 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0879e-4cb3-49ac-bfda-769103fd9984 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:29:47.055552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-6da0879e-4cb3-49ac-bfda-769103fd9984 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:29:47.135115 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0879e-4cb3-49ac-bfda-769103fd9984 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:29:47.139470 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8653eb-94f7-4473-9c94-9d25408e7079 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:29:47.141144 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8653eb-94f7-4473-9c94-9d25408e7079 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:29:47.141510 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8653eb-94f7-4473-9c94-9d25408e7079 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:29:47.141789 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8653eb-94f7-4473-9c94-9d25408e7079 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:29:47.159715 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8653eb-94f7-4473-9c94-9d25408e7079 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:29:47.161525 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8653eb-94f7-4473-9c94-9d25408e7079 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:29:47.172765 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1197/2364] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:29:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 06:29:57.100920 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=31 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:29:57.167663 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=60,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:30:05.214224 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75de6b-2454-41e8-89f0-ed6779994531 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:30:05.214844 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75de6b-2454-41e8-89f0-ed6779994531 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:30:05.214844 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75de6b-2454-41e8-89f0-ed6779994531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:30:05.245511 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-6c75de6b-2454-41e8-89f0-ed6779994531 None None] MFA Rules not processed for user `621a7ea4f60545d5a5c30b2b7a370bf0`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:30:05.247663 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-6c75de6b-2454-41e8-89f0-ed6779994531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:30:05.337679 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1168/2365] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:30:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:30:15.323528 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=25 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:30:16.395778 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa29ed5-5581-44a5-b804-d3c1de002a80 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:30:16.396714 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa29ed5-5581-44a5-b804-d3c1de002a80 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:30:16.396890 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa29ed5-5581-44a5-b804-d3c1de002a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:30:16.416840 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-faa29ed5-5581-44a5-b804-d3c1de002a80 None None] MFA Rules not processed for user `c75ed30649124403b3f1f2f8bdc6a377`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:30:16.418203 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-faa29ed5-5581-44a5-b804-d3c1de002a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:30:16.496465 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1198/2366] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:30:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:30:21.759175 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75de6b-2454-41e8-89f0-ed6779994531 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:30:21.834418 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75de6b-2454-41e8-89f0-ed6779994531 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:30:21.839217 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff32691-55b1-4cb9-9d73-b06f1788e3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:30:21.840138 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff32691-55b1-4cb9-9d73-b06f1788e3e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:30:21.840444 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff32691-55b1-4cb9-9d73-b06f1788e3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:30:21.840738 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff32691-55b1-4cb9-9d73-b06f1788e3e4 admin admin] PATH_INFO: `/v3/users/f7c2c5faf28b43b3b879a267b807c006` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:30:21.842318 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff32691-55b1-4cb9-9d73-b06f1788e3e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7c2c5faf28b43b3b879a267b807c006)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:30:21.844443 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff32691-55b1-4cb9-9d73-b06f1788e3e4 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:30:22.089939 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-8ff32691-55b1-4cb9-9d73-b06f1788e3e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7c2c5faf28b43b3b879a267b807c006'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:30:22.099620 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-8ff32691-55b1-4cb9-9d73-b06f1788e3e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7c2c5faf28b43b3b879a267b807c006'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:30:22.128134 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-8ff32691-55b1-4cb9-9d73-b06f1788e3e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7c2c5faf28b43b3b879a267b807c006'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:30:22.128452 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-8ff32691-55b1-4cb9-9d73-b06f1788e3e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7c2c5faf28b43b3b879a267b807c006'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:30:22.130641 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-8ff32691-55b1-4cb9-9d73-b06f1788e3e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7c2c5faf28b43b3b879a267b807c006'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:30:22.218250 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1169/2367] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:30:21 2022] DELETE /identity/v3/users/f7c2c5faf28b43b3b879a267b807c006 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:30:22.224301 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbfa135-d2b9-4ff8-ba17-9e8b33659c60 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:30:22.227770 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbfa135-d2b9-4ff8-ba17-9e8b33659c60 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:30:22.227770 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbfa135-d2b9-4ff8-ba17-9e8b33659c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:30:22.245807 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-bfbfa135-d2b9-4ff8-ba17-9e8b33659c60 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:30:22.247050 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-bfbfa135-d2b9-4ff8-ba17-9e8b33659c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:30:22.312273 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1199/2368] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:30:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:30:22.322916 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff32691-55b1-4cb9-9d73-b06f1788e3e4 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:30:22.325724 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8ff32691-55b1-4cb9-9d73-b06f1788e3e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:30:22.407885 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff32691-55b1-4cb9-9d73-b06f1788e3e4 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:30:22.412334 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f07de-95ac-4621-9d5b-c977596e45df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:30:22.413361 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f07de-95ac-4621-9d5b-c977596e45df service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:30:22.413780 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f07de-95ac-4621-9d5b-c977596e45df service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:30:22.414201 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f07de-95ac-4621-9d5b-c977596e45df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:30:22.428447 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f07de-95ac-4621-9d5b-c977596e45df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:30:22.428447 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f07de-95ac-4621-9d5b-c977596e45df service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:30:22.446511 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1170/2369] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:30:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:30:23.313828 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbfa135-d2b9-4ff8-ba17-9e8b33659c60 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:30:23.315412 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-bfbfa135-d2b9-4ff8-ba17-9e8b33659c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:30:23.327975 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbfa135-d2b9-4ff8-ba17-9e8b33659c60 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:30:23.334510 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a776e6b1-224a-4352-bf78-c2ffa44603d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:30:23.335444 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776e6b1-224a-4352-bf78-c2ffa44603d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:30:23.335737 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776e6b1-224a-4352-bf78-c2ffa44603d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:30:23.336023 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776e6b1-224a-4352-bf78-c2ffa44603d6 admin admin] PATH_INFO: `/v3/projects/47260a387394403082e8d925797c6285` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:30:23.337770 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a776e6b1-224a-4352-bf78-c2ffa44603d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47260a387394403082e8d925797c6285)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:30:23.339321 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a776e6b1-224a-4352-bf78-c2ffa44603d6 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:30:23.533486 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-a776e6b1-224a-4352-bf78-c2ffa44603d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47260a387394403082e8d925797c6285', 'request_id': 'req-a776e6b1-224a-4352-bf78-c2ffa44603d6'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:30:23.542705 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-a776e6b1-224a-4352-bf78-c2ffa44603d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47260a387394403082e8d925797c6285', 'request_id': 'req-a776e6b1-224a-4352-bf78-c2ffa44603d6'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:30:23.668003 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-a776e6b1-224a-4352-bf78-c2ffa44603d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47260a387394403082e8d925797c6285', 'request_id': 'req-a776e6b1-224a-4352-bf78-c2ffa44603d6'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:30:23.692344 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-a776e6b1-224a-4352-bf78-c2ffa44603d6 admin admin] The token cache is being invalidate because project 47260a387394403082e8d925797c6285 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:30:23.692757 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-a776e6b1-224a-4352-bf78-c2ffa44603d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:30:23.693680 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-a776e6b1-224a-4352-bf78-c2ffa44603d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:30:23.700190 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1200/2370] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:30:23 2022] DELETE /identity/v3/projects/47260a387394403082e8d925797c6285 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:30:23.702432 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c5a394-d381-4b9a-bf7c-ccfabd3f6198 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:30:23.702970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c5a394-d381-4b9a-bf7c-ccfabd3f6198 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:30:23.703464 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c5a394-d381-4b9a-bf7c-ccfabd3f6198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:30:23.741998 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-55c5a394-d381-4b9a-bf7c-ccfabd3f6198 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:30:23.744395 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-55c5a394-d381-4b9a-bf7c-ccfabd3f6198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:30:23.780878 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:30:23.864514 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1171/2371] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:30:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:30:23.872945 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07da787-7156-41f3-acc7-95f414fe1952 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:30:23.873431 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07da787-7156-41f3-acc7-95f414fe1952 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:30:23.873912 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07da787-7156-41f3-acc7-95f414fe1952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:30:23.896846 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-e07da787-7156-41f3-acc7-95f414fe1952 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:30:23.898202 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e07da787-7156-41f3-acc7-95f414fe1952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:30:23.908925 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1201/2372] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:30:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:30:23.913410 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c5a394-d381-4b9a-bf7c-ccfabd3f6198 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:30:23.932651 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c5a394-d381-4b9a-bf7c-ccfabd3f6198 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:30:23.939190 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e87afec-f4c4-4eb1-b94b-3edcbfb6f2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:30:23.940040 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87afec-f4c4-4eb1-b94b-3edcbfb6f2fa admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:30:23.940395 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87afec-f4c4-4eb1-b94b-3edcbfb6f2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:30:23.940781 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87afec-f4c4-4eb1-b94b-3edcbfb6f2fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:30:23.941617 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e87afec-f4c4-4eb1-b94b-3edcbfb6f2fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:30:23.942021 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e87afec-f4c4-4eb1-b94b-3edcbfb6f2fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:30:23.943798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e87afec-f4c4-4eb1-b94b-3edcbfb6f2fa admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:30:23.964939 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1172/2373] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:30:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:30:23.965058 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07da787-7156-41f3-acc7-95f414fe1952 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:30:23.973827 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07da787-7156-41f3-acc7-95f414fe1952 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:30:23.991831 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa992eab-7c36-4c00-a504-2d2a10d32237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:30:23.993340 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa992eab-7c36-4c00-a504-2d2a10d32237 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:30:23.994676 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa992eab-7c36-4c00-a504-2d2a10d32237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:30:23.995296 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa992eab-7c36-4c00-a504-2d2a10d32237 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:30:23.996773 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa992eab-7c36-4c00-a504-2d2a10d32237 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:30:24.001395 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa992eab-7c36-4c00-a504-2d2a10d32237 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:30:24.091501 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:30:24.100135 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1202/2374] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:30:23 2022] POST /identity/v3/projects => generated 399 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:30:24.105382 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e87afec-f4c4-4eb1-b94b-3edcbfb6f2fa admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:30:24.118523 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e87afec-f4c4-4eb1-b94b-3edcbfb6f2fa admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:30:24.186603 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502bdb4f-ed4e-4497-bfd1-f44f6c404364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:30:24.187358 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502bdb4f-ed4e-4497-bfd1-f44f6c404364 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:30:24.187721 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502bdb4f-ed4e-4497-bfd1-f44f6c404364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:30:24.188004 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502bdb4f-ed4e-4497-bfd1-f44f6c404364 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:30:24.189999 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502bdb4f-ed4e-4497-bfd1-f44f6c404364 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:30:24.192288 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502bdb4f-ed4e-4497-bfd1-f44f6c404364 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:30:24.347935 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1173/2375] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:30:24 2022] POST /identity/v3/users => generated 300 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:30:24.348340 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa992eab-7c36-4c00-a504-2d2a10d32237 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:30:24.366967 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa992eab-7c36-4c00-a504-2d2a10d32237 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:30:24.374380 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b0a33-5764-4142-a015-ce3915e09cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:30:24.375429 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b0a33-5764-4142-a015-ce3915e09cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:30:24.376204 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b0a33-5764-4142-a015-ce3915e09cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:30:24.377483 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b0a33-5764-4142-a015-ce3915e09cd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:30:24.378708 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239b0a33-5764-4142-a015-ce3915e09cd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:30:24.380740 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239b0a33-5764-4142-a015-ce3915e09cd4 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:30:24.399485 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1203/2376] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:30:24.404892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502bdb4f-ed4e-4497-bfd1-f44f6c404364 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:30:24.415845 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502bdb4f-ed4e-4497-bfd1-f44f6c404364 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:30:24.421484 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf51f0-5cda-4b61-aa19-2a1c0a682072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:30:24.422335 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf51f0-5cda-4b61-aa19-2a1c0a682072 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:30:24.422627 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf51f0-5cda-4b61-aa19-2a1c0a682072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:30:24.423184 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf51f0-5cda-4b61-aa19-2a1c0a682072 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:30:24.424094 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bf51f0-5cda-4b61-aa19-2a1c0a682072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:30:24.425826 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bf51f0-5cda-4b61-aa19-2a1c0a682072 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:30:24.440054 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1174/2377] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:30:24.447572 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b0a33-5764-4142-a015-ce3915e09cd4 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:30:24.467189 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b0a33-5764-4142-a015-ce3915e09cd4 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:30:24.473283 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f9b58-9a7c-42a2-9501-56db9e356854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:30:24.474310 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f9b58-9a7c-42a2-9501-56db9e356854 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:30:24.474590 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f9b58-9a7c-42a2-9501-56db9e356854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:30:24.474869 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f9b58-9a7c-42a2-9501-56db9e356854 admin admin] PATH_INFO: `/v3/projects/4475f07692d441fd83d5d8ac4b123fb2/users/de0888b6263049a9a688f8ed6bdb30b8/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:30:24.491810 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366f9b58-9a7c-42a2-9501-56db9e356854 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4475f07692d441fd83d5d8ac4b123fb2, user_id=de0888b6263049a9a688f8ed6bdb30b8, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:30:24.495440 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366f9b58-9a7c-42a2-9501-56db9e356854 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:30:24.588793 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1204/2378] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:30:24 2022] PUT /identity/v3/projects/4475f07692d441fd83d5d8ac4b123fb2/users/de0888b6263049a9a688f8ed6bdb30b8/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:30:24.603882 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa1eaac-7d9a-4c25-948c-c70d125d6c1d None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:30:24.604227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa1eaac-7d9a-4c25-948c-c70d125d6c1d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:30:24.604668 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa1eaac-7d9a-4c25-948c-c70d125d6c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:30:24.644253 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-dfa1eaac-7d9a-4c25-948c-c70d125d6c1d None None] MFA Rules not processed for user `de0888b6263049a9a688f8ed6bdb30b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:30:24.645630 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-dfa1eaac-7d9a-4c25-948c-c70d125d6c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:30:24.763054 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1175/2379] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:30:24 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:30:24.775142 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f9b58-9a7c-42a2-9501-56db9e356854 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:30:24.777053 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-366f9b58-9a7c-42a2-9501-56db9e356854 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:30:24.858335 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f9b58-9a7c-42a2-9501-56db9e356854 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:30:24.862759 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b11687-7f5e-4f01-bc9a-5bfc27ac0cbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:30:24.863420 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b11687-7f5e-4f01-bc9a-5bfc27ac0cbe service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:30:24.863706 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b11687-7f5e-4f01-bc9a-5bfc27ac0cbe service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:30:24.863984 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b11687-7f5e-4f01-bc9a-5bfc27ac0cbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:30:24.873474 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b11687-7f5e-4f01-bc9a-5bfc27ac0cbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:30:24.876091 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b11687-7f5e-4f01-bc9a-5bfc27ac0cbe service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:30:24.886431 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1205/2380] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:30:24 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 06:30:34.760124 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:30:34.889948 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:30:36.357333 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa1eaac-7d9a-4c25-948c-c70d125d6c1d None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:30:36.446923 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa1eaac-7d9a-4c25-948c-c70d125d6c1d None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:30:36.452032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2844f9-d524-410c-9fcf-778ddacd519d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:30:36.453040 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2844f9-d524-410c-9fcf-778ddacd519d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:30:36.453324 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2844f9-d524-410c-9fcf-778ddacd519d admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:30:36.454337 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2844f9-d524-410c-9fcf-778ddacd519d admin admin] PATH_INFO: `/v3/users/de0888b6263049a9a688f8ed6bdb30b8` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:30:36.455905 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2844f9-d524-410c-9fcf-778ddacd519d admin admin] RBAC: Authorizing `identity:delete_user(user_id=de0888b6263049a9a688f8ed6bdb30b8)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:30:36.457997 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2844f9-d524-410c-9fcf-778ddacd519d admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:30:36.672186 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-4c2844f9-d524-410c-9fcf-778ddacd519d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de0888b6263049a9a688f8ed6bdb30b8'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:30:36.680604 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-4c2844f9-d524-410c-9fcf-778ddacd519d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de0888b6263049a9a688f8ed6bdb30b8'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:30:36.699535 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-4c2844f9-d524-410c-9fcf-778ddacd519d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de0888b6263049a9a688f8ed6bdb30b8'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:30:36.700288 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-4c2844f9-d524-410c-9fcf-778ddacd519d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de0888b6263049a9a688f8ed6bdb30b8'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:30:36.700967 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-4c2844f9-d524-410c-9fcf-778ddacd519d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de0888b6263049a9a688f8ed6bdb30b8'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:30:36.767744 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1176/2381] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:30:36 2022] DELETE /identity/v3/users/de0888b6263049a9a688f8ed6bdb30b8 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:30:36.772811 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef99b13-1aac-4177-b268-8a2a137aa308 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:30:36.773355 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef99b13-1aac-4177-b268-8a2a137aa308 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:30:36.773547 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef99b13-1aac-4177-b268-8a2a137aa308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:30:36.795618 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-4ef99b13-1aac-4177-b268-8a2a137aa308 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:30:36.796988 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4ef99b13-1aac-4177-b268-8a2a137aa308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:30:36.868354 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1206/2382] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:30:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:30:36.878993 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2844f9-d524-410c-9fcf-778ddacd519d admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:30:36.880610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4c2844f9-d524-410c-9fcf-778ddacd519d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:30:36.949800 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2844f9-d524-410c-9fcf-778ddacd519d admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:30:36.953717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f289f65-4bee-4177-8f7a-c9721b225207 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:30:36.960350 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f289f65-4bee-4177-8f7a-c9721b225207 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:30:36.960350 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f289f65-4bee-4177-8f7a-c9721b225207 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:30:36.960350 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f289f65-4bee-4177-8f7a-c9721b225207 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:30:36.967151 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f289f65-4bee-4177-8f7a-c9721b225207 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:30:36.968720 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f289f65-4bee-4177-8f7a-c9721b225207 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:30:36.980023 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1177/2383] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:30:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:30:37.836165 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef99b13-1aac-4177-b268-8a2a137aa308 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:30:37.839339 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4ef99b13-1aac-4177-b268-8a2a137aa308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:30:37.854525 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef99b13-1aac-4177-b268-8a2a137aa308 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:30:37.859809 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe9642-ccb6-4e6c-b548-559fb565d522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:30:37.861562 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe9642-ccb6-4e6c-b548-559fb565d522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:30:37.861877 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe9642-ccb6-4e6c-b548-559fb565d522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:30:37.862109 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe9642-ccb6-4e6c-b548-559fb565d522 admin admin] PATH_INFO: `/v3/projects/4475f07692d441fd83d5d8ac4b123fb2` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:30:37.863676 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efe9642-ccb6-4e6c-b548-559fb565d522 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4475f07692d441fd83d5d8ac4b123fb2)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:30:37.865847 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efe9642-ccb6-4e6c-b548-559fb565d522 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:30:38.077797 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8efe9642-ccb6-4e6c-b548-559fb565d522 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4475f07692d441fd83d5d8ac4b123fb2', 'request_id': 'req-8efe9642-ccb6-4e6c-b548-559fb565d522'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:30:38.082367 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8efe9642-ccb6-4e6c-b548-559fb565d522 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4475f07692d441fd83d5d8ac4b123fb2', 'request_id': 'req-8efe9642-ccb6-4e6c-b548-559fb565d522'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:30:38.181117 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8efe9642-ccb6-4e6c-b548-559fb565d522 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4475f07692d441fd83d5d8ac4b123fb2', 'request_id': 'req-8efe9642-ccb6-4e6c-b548-559fb565d522'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:30:38.193224 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8efe9642-ccb6-4e6c-b548-559fb565d522 admin admin] The token cache is being invalidate because project 4475f07692d441fd83d5d8ac4b123fb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:30:38.193593 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8efe9642-ccb6-4e6c-b548-559fb565d522 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:30:38.194255 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8efe9642-ccb6-4e6c-b548-559fb565d522 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:30:38.195917 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1207/2384] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:30:37 2022] DELETE /identity/v3/projects/4475f07692d441fd83d5d8ac4b123fb2 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:30:44.151820 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f289f65-4bee-4177-8f7a-c9721b225207 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:30:44.161044 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f289f65-4bee-4177-8f7a-c9721b225207 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:30:44.187999 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:30:44.227051 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82a3ba-0247-4d7b-8c82-c2792482d5d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:30:44.228541 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82a3ba-0247-4d7b-8c82-c2792482d5d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:30:44.229040 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82a3ba-0247-4d7b-8c82-c2792482d5d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:30:44.229523 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82a3ba-0247-4d7b-8c82-c2792482d5d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:30:44.232862 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-9f82a3ba-0247-4d7b-8c82-c2792482d5d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:30:44.329124 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f82a3ba-0247-4d7b-8c82-c2792482d5d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:30:44.330481 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f82a3ba-0247-4d7b-8c82-c2792482d5d0 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:30:44.341839 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1178/2385] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:30:44 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:30:48.190215 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:30:53.947734 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe877478-a080-4b57-b429-9c527df19b43 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:30:53.948334 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe877478-a080-4b57-b429-9c527df19b43 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:30:53.948334 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe877478-a080-4b57-b429-9c527df19b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:30:53.966518 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-fe877478-a080-4b57-b429-9c527df19b43 None None] MFA Rules not processed for user `b84dde57ff8e47419866377a819bea21`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:30:53.967733 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-fe877478-a080-4b57-b429-9c527df19b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:30:54.029822 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1208/2386] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Jul 18 06:30:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:30:54.169254 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82a3ba-0247-4d7b-8c82-c2792482d5d0 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:30:54.170817 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-9f82a3ba-0247-4d7b-8c82-c2792482d5d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:30:54.255769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82a3ba-0247-4d7b-8c82-c2792482d5d0 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:30:54.261693 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc34820-5700-427d-9b02-55959c3ceca5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0681f5fbea6d415c9ad2f3afae256a5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:30:54.262552 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc34820-5700-427d-9b02-55959c3ceca5 service swift] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:30:54.262944 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc34820-5700-427d-9b02-55959c3ceca5 service swift] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:30:54.263306 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc34820-5700-427d-9b02-55959c3ceca5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:30:54.277819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc34820-5700-427d-9b02-55959c3ceca5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:30:54.277819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc34820-5700-427d-9b02-55959c3ceca5 service swift] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:30:54.286727 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1179/2387] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Jul 18 06:30:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 06:31:04.022701 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=25 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:31:04.282094 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=72 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:31:06.566253 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe877478-a080-4b57-b429-9c527df19b43 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:31:06.570611 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-fe877478-a080-4b57-b429-9c527df19b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:31:06.595985 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe877478-a080-4b57-b429-9c527df19b43 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:31:06.602399 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5213c326-c869-439e-ac8f-571ad8dd7fbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:31:06.603534 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213c326-c869-439e-ac8f-571ad8dd7fbc service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:31:06.604020 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213c326-c869-439e-ac8f-571ad8dd7fbc service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:31:06.604520 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213c326-c869-439e-ac8f-571ad8dd7fbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:31:06.606536 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-5213c326-c869-439e-ac8f-571ad8dd7fbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:31:06.752591 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5213c326-c869-439e-ac8f-571ad8dd7fbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:31:06.754188 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5213c326-c869-439e-ac8f-571ad8dd7fbc service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:31:06.770027 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1209/2388] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:31:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:31:08.014350 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc34820-5700-427d-9b02-55959c3ceca5 service swift] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:31:08.029075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc34820-5700-427d-9b02-55959c3ceca5 service swift] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:31:08.033863 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db72106-6384-48bf-b482-63ae6ef29042 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:31:08.034880 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db72106-6384-48bf-b482-63ae6ef29042 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:31:08.035316 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db72106-6384-48bf-b482-63ae6ef29042 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:31:08.037704 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db72106-6384-48bf-b482-63ae6ef29042 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:31:08.040560 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-2db72106-6384-48bf-b482-63ae6ef29042 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:31:08.056727 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db72106-6384-48bf-b482-63ae6ef29042 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:31:08.058478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db72106-6384-48bf-b482-63ae6ef29042 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:31:08.073472 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1180/2389] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:31:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:31:11.651921 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d0a9dc-5204-4db8-aa52-a55cd410c89f None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:31:11.652703 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d0a9dc-5204-4db8-aa52-a55cd410c89f None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:31:11.653007 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d0a9dc-5204-4db8-aa52-a55cd410c89f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:31:11.684178 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-a4d0a9dc-5204-4db8-aa52-a55cd410c89f None None] MFA Rules not processed for user `b84dde57ff8e47419866377a819bea21`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:31:11.686486 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-a4d0a9dc-5204-4db8-aa52-a55cd410c89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:31:11.700151 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1210/2390] 10.222.0.40 () {58 vars in 967 bytes} [Mon Jul 18 06:31:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:31:11.713321 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db72106-6384-48bf-b482-63ae6ef29042 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:31:11.726101 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db72106-6384-48bf-b482-63ae6ef29042 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:31:11.731568 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99117cfb-9359-4c7e-abc9-427b4e361cb6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0681f5fbea6d415c9ad2f3afae256a5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:31:11.732237 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99117cfb-9359-4c7e-abc9-427b4e361cb6 service swift] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:31:11.732567 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99117cfb-9359-4c7e-abc9-427b4e361cb6 service swift] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:31:11.732852 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99117cfb-9359-4c7e-abc9-427b4e361cb6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:31:11.744336 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99117cfb-9359-4c7e-abc9-427b4e361cb6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:31:11.746282 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99117cfb-9359-4c7e-abc9-427b4e361cb6 service swift] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:31:11.757113 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1181/2391] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Jul 18 06:31:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 06:31:15.162063 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d0a9dc-5204-4db8-aa52-a55cd410c89f None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:31:15.163679 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-a4d0a9dc-5204-4db8-aa52-a55cd410c89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:31:15.301501 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d0a9dc-5204-4db8-aa52-a55cd410c89f None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:31:15.309638 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87073434-7fbc-49bc-b87a-373535dbc52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:31:15.311147 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87073434-7fbc-49bc-b87a-373535dbc52a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:31:15.311626 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87073434-7fbc-49bc-b87a-373535dbc52a admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:31:15.312105 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87073434-7fbc-49bc-b87a-373535dbc52a admin admin] PATH_INFO: `/v3/users/c75ed30649124403b3f1f2f8bdc6a377` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:31:15.314319 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87073434-7fbc-49bc-b87a-373535dbc52a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c75ed30649124403b3f1f2f8bdc6a377)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:31:15.319539 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87073434-7fbc-49bc-b87a-373535dbc52a admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:31:15.522108 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-87073434-7fbc-49bc-b87a-373535dbc52a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c75ed30649124403b3f1f2f8bdc6a377'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:31:15.530881 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-87073434-7fbc-49bc-b87a-373535dbc52a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c75ed30649124403b3f1f2f8bdc6a377'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:31:15.531630 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-87073434-7fbc-49bc-b87a-373535dbc52a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c75ed30649124403b3f1f2f8bdc6a377'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:31:15.532253 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-87073434-7fbc-49bc-b87a-373535dbc52a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c75ed30649124403b3f1f2f8bdc6a377'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:31:15.550690 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-87073434-7fbc-49bc-b87a-373535dbc52a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c75ed30649124403b3f1f2f8bdc6a377'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:31:15.627882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:31:15.634620 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1211/2392] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:31:15 2022] DELETE /identity/v3/users/c75ed30649124403b3f1f2f8bdc6a377 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:31:15.962998 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99117cfb-9359-4c7e-abc9-427b4e361cb6 service swift] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:31:15.981123 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99117cfb-9359-4c7e-abc9-427b4e361cb6 service swift] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:31:16.061094 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a4d96-2b57-4971-b871-eef4c6c7dca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:31:16.061925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a4d96-2b57-4971-b871-eef4c6c7dca2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:31:16.062214 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a4d96-2b57-4971-b871-eef4c6c7dca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:31:16.062482 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a4d96-2b57-4971-b871-eef4c6c7dca2 admin admin] PATH_INFO: `/v3/projects/123fd072dc8b4ac095c1ee3579798e4b` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:31:16.063717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807a4d96-2b57-4971-b871-eef4c6c7dca2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=123fd072dc8b4ac095c1ee3579798e4b)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:31:16.065712 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807a4d96-2b57-4971-b871-eef4c6c7dca2 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:31:16.424284 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-807a4d96-2b57-4971-b871-eef4c6c7dca2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '123fd072dc8b4ac095c1ee3579798e4b', 'request_id': 'req-807a4d96-2b57-4971-b871-eef4c6c7dca2'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:31:16.430691 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-807a4d96-2b57-4971-b871-eef4c6c7dca2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '123fd072dc8b4ac095c1ee3579798e4b', 'request_id': 'req-807a4d96-2b57-4971-b871-eef4c6c7dca2'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:31:16.512909 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-807a4d96-2b57-4971-b871-eef4c6c7dca2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '123fd072dc8b4ac095c1ee3579798e4b', 'request_id': 'req-807a4d96-2b57-4971-b871-eef4c6c7dca2'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:31:16.527271 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-807a4d96-2b57-4971-b871-eef4c6c7dca2 admin admin] The token cache is being invalidate because project 123fd072dc8b4ac095c1ee3579798e4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:31:16.527591 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-807a4d96-2b57-4971-b871-eef4c6c7dca2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:31:16.528254 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-807a4d96-2b57-4971-b871-eef4c6c7dca2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:31:16.529876 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1182/2393] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:31:15 2022] DELETE /identity/v3/projects/123fd072dc8b4ac095c1ee3579798e4b => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:31:16.535982 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460fa42-4210-4c57-8456-8bb828824d2a None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:31:16.536598 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460fa42-4210-4c57-8456-8bb828824d2a None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:31:16.536817 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460fa42-4210-4c57-8456-8bb828824d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:31:16.556752 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-5460fa42-4210-4c57-8456-8bb828824d2a None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:31:16.558085 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-5460fa42-4210-4c57-8456-8bb828824d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:31:16.632307 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1212/2394] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:31:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:31:16.640886 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69e1dd-1e12-4e09-999a-53cfb2b1eb31 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:31:16.641283 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69e1dd-1e12-4e09-999a-53cfb2b1eb31 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:31:16.641637 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69e1dd-1e12-4e09-999a-53cfb2b1eb31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:31:16.663280 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-4e69e1dd-1e12-4e09-999a-53cfb2b1eb31 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:31:16.664581 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4e69e1dd-1e12-4e09-999a-53cfb2b1eb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:31:16.674439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1183/2395] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:31:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:31:16.679997 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5460fa42-4210-4c57-8456-8bb828824d2a None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:31:16.691254 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5460fa42-4210-4c57-8456-8bb828824d2a None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:31:16.696617 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf018fde-d18c-44aa-a02f-c1148871e45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:31:16.697321 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf018fde-d18c-44aa-a02f-c1148871e45c admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:31:16.697654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf018fde-d18c-44aa-a02f-c1148871e45c admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:31:16.697955 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf018fde-d18c-44aa-a02f-c1148871e45c admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:31:16.698751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf018fde-d18c-44aa-a02f-c1148871e45c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:31:16.699102 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf018fde-d18c-44aa-a02f-c1148871e45c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:31:16.701073 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf018fde-d18c-44aa-a02f-c1148871e45c admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:31:16.715902 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1213/2396] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:31:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:31:16.721073 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e69e1dd-1e12-4e09-999a-53cfb2b1eb31 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:31:16.739790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e69e1dd-1e12-4e09-999a-53cfb2b1eb31 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:31:16.747863 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd53fd8-9542-4481-8f9e-eb1c07a129c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:31:16.749056 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd53fd8-9542-4481-8f9e-eb1c07a129c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:31:16.749539 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd53fd8-9542-4481-8f9e-eb1c07a129c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:31:16.750225 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd53fd8-9542-4481-8f9e-eb1c07a129c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:31:16.751569 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd53fd8-9542-4481-8f9e-eb1c07a129c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:31:16.755171 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd53fd8-9542-4481-8f9e-eb1c07a129c3 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:31:16.887245 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1184/2397] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:31:16 2022] POST /identity/v3/projects => generated 383 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:31:16.891406 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf018fde-d18c-44aa-a02f-c1148871e45c admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:31:16.903625 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf018fde-d18c-44aa-a02f-c1148871e45c admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:31:16.972882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea7f4c-5702-4261-ba63-daa06ba84da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:31:16.974119 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea7f4c-5702-4261-ba63-daa06ba84da0 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:31:16.974437 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea7f4c-5702-4261-ba63-daa06ba84da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:31:16.974701 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea7f4c-5702-4261-ba63-daa06ba84da0 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:31:16.975428 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ea7f4c-5702-4261-ba63-daa06ba84da0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:31:16.977442 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ea7f4c-5702-4261-ba63-daa06ba84da0 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:31:17.091513 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1214/2398] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:31:16 2022] POST /identity/v3/users => generated 292 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:31:17.096094 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd53fd8-9542-4481-8f9e-eb1c07a129c3 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:31:17.138510 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd53fd8-9542-4481-8f9e-eb1c07a129c3 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:31:17.145503 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33139bf7-a9a9-424c-8370-e54189e54375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:31:17.146400 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33139bf7-a9a9-424c-8370-e54189e54375 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:31:17.146804 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33139bf7-a9a9-424c-8370-e54189e54375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:31:17.147082 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33139bf7-a9a9-424c-8370-e54189e54375 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:31:17.147842 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33139bf7-a9a9-424c-8370-e54189e54375 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:31:17.149508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33139bf7-a9a9-424c-8370-e54189e54375 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:31:17.164767 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1185/2399] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 06:31:17.169510 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea7f4c-5702-4261-ba63-daa06ba84da0 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:31:17.181443 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea7f4c-5702-4261-ba63-daa06ba84da0 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:31:17.186601 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912e886-615b-4bbe-87ed-11e61f6553b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:31:17.187296 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912e886-615b-4bbe-87ed-11e61f6553b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:31:17.187617 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912e886-615b-4bbe-87ed-11e61f6553b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:31:17.187918 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912e886-615b-4bbe-87ed-11e61f6553b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:31:17.188779 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9912e886-615b-4bbe-87ed-11e61f6553b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:31:17.190617 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9912e886-615b-4bbe-87ed-11e61f6553b1 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:31:17.202890 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1215/2400] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:31:17.209805 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33139bf7-a9a9-424c-8370-e54189e54375 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:31:17.220696 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33139bf7-a9a9-424c-8370-e54189e54375 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:31:17.227881 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a42b5b-8699-4241-aabb-6872a728c5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:31:17.229425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a42b5b-8699-4241-aabb-6872a728c5ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:31:17.229960 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a42b5b-8699-4241-aabb-6872a728c5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:31:17.230508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a42b5b-8699-4241-aabb-6872a728c5ba admin admin] PATH_INFO: `/v3/projects/c7a3d0bcdbd24e498ab3664983af9298/users/81d9b70897ad4f8fabe72d1728558181/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:31:17.250246 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a42b5b-8699-4241-aabb-6872a728c5ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7a3d0bcdbd24e498ab3664983af9298, user_id=81d9b70897ad4f8fabe72d1728558181, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:31:17.253001 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a42b5b-8699-4241-aabb-6872a728c5ba admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:31:17.336587 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:31:17.341992 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1186/2401] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:31:17 2022] PUT /identity/v3/projects/c7a3d0bcdbd24e498ab3664983af9298/users/81d9b70897ad4f8fabe72d1728558181/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:31:17.347691 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02609522-f45f-440d-b396-ccb911697435 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:31:17.348512 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02609522-f45f-440d-b396-ccb911697435 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:31:17.348895 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02609522-f45f-440d-b396-ccb911697435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:31:17.370955 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-02609522-f45f-440d-b396-ccb911697435 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:31:17.372187 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-02609522-f45f-440d-b396-ccb911697435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:31:17.452663 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1216/2402] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:31:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:31:17.464987 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a42b5b-8699-4241-aabb-6872a728c5ba admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:31:17.467700 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-34a42b5b-8699-4241-aabb-6872a728c5ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:31:17.543511 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a42b5b-8699-4241-aabb-6872a728c5ba admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:31:17.547247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a24d3-aa18-4072-876a-cf31d9b6eb1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:31:17.547905 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a24d3-aa18-4072-876a-cf31d9b6eb1c service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:31:17.548191 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a24d3-aa18-4072-876a-cf31d9b6eb1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:31:17.548490 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a24d3-aa18-4072-876a-cf31d9b6eb1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:31:17.559881 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a24d3-aa18-4072-876a-cf31d9b6eb1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:31:17.561376 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a24d3-aa18-4072-876a-cf31d9b6eb1c service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:31:17.572576 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1187/2403] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:31:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:31:24.084720 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538de207-2c06-4d83-a134-716d116af74f None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:31:24.084720 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538de207-2c06-4d83-a134-716d116af74f None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:31:24.084720 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538de207-2c06-4d83-a134-716d116af74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:31:24.084720 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:31:24.136898 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-538de207-2c06-4d83-a134-716d116af74f None None] MFA Rules not processed for user `81d9b70897ad4f8fabe72d1728558181`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:31:24.136898 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-538de207-2c06-4d83-a134-716d116af74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:31:24.224393 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1217/2404] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:31:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:31:24.228815 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a24d3-aa18-4072-876a-cf31d9b6eb1c service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:31:24.240295 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a24d3-aa18-4072-876a-cf31d9b6eb1c service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:31:24.248542 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49c7702-6dfe-46fc-9576-e455a42cebca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:31:24.248918 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c7702-6dfe-46fc-9576-e455a42cebca admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:31:24.248918 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c7702-6dfe-46fc-9576-e455a42cebca admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:31:24.249241 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c7702-6dfe-46fc-9576-e455a42cebca admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:31:24.249913 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49c7702-6dfe-46fc-9576-e455a42cebca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:31:24.251834 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49c7702-6dfe-46fc-9576-e455a42cebca admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:31:24.364950 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1188/2405] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:31:24 2022] POST /identity/v3/projects => generated 385 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:31:24.366185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538de207-2c06-4d83-a134-716d116af74f None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:31:24.378268 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538de207-2c06-4d83-a134-716d116af74f None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:31:24.455493 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47db9a5c-1b69-42a2-ab8b-09c467f16850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:31:24.456196 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db9a5c-1b69-42a2-ab8b-09c467f16850 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:31:24.456516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db9a5c-1b69-42a2-ab8b-09c467f16850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:31:24.456795 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db9a5c-1b69-42a2-ab8b-09c467f16850 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:31:24.458041 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47db9a5c-1b69-42a2-ab8b-09c467f16850 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:31:24.460303 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47db9a5c-1b69-42a2-ab8b-09c467f16850 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:31:24.556910 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1218/2406] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:31:24 2022] POST /identity/v3/users => generated 299 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:31:24.563317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49c7702-6dfe-46fc-9576-e455a42cebca admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:31:24.581111 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49c7702-6dfe-46fc-9576-e455a42cebca admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:31:24.593012 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c639b95-421a-4c09-a36e-b023892a6d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:31:24.593012 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c639b95-421a-4c09-a36e-b023892a6d23 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:31:24.593648 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c639b95-421a-4c09-a36e-b023892a6d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:31:24.594023 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c639b95-421a-4c09-a36e-b023892a6d23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:31:24.595348 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c639b95-421a-4c09-a36e-b023892a6d23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:31:24.599008 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c639b95-421a-4c09-a36e-b023892a6d23 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:31:24.623332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1189/2407] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:31:24.624974 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47db9a5c-1b69-42a2-ab8b-09c467f16850 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:31:24.644089 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47db9a5c-1b69-42a2-ab8b-09c467f16850 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:31:24.676812 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c8835-d6db-467c-a0d9-16438f92e148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:31:24.677838 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c8835-d6db-467c-a0d9-16438f92e148 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:31:24.678239 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c8835-d6db-467c-a0d9-16438f92e148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:31:24.678619 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c8835-d6db-467c-a0d9-16438f92e148 admin admin] PATH_INFO: `/v3/projects/0e44d69624be4720b0151f6cd16cde34/users/d29d4b602518487cb496bb26c6bf9780/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:31:24.699263 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818c8835-d6db-467c-a0d9-16438f92e148 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e44d69624be4720b0151f6cd16cde34, user_id=d29d4b602518487cb496bb26c6bf9780, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:31:24.703242 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818c8835-d6db-467c-a0d9-16438f92e148 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:31:24.811639 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1219/2408] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:31:24 2022] PUT /identity/v3/projects/0e44d69624be4720b0151f6cd16cde34/users/d29d4b602518487cb496bb26c6bf9780/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:31:24.820281 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c639b95-421a-4c09-a36e-b023892a6d23 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:31:24.850541 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c639b95-421a-4c09-a36e-b023892a6d23 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:31:24.944274 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0181889-714e-432a-ab06-4950ed72b509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:31:24.945125 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0181889-714e-432a-ab06-4950ed72b509 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:31:24.945896 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0181889-714e-432a-ab06-4950ed72b509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:31:24.945896 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0181889-714e-432a-ab06-4950ed72b509 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:31:24.946900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0181889-714e-432a-ab06-4950ed72b509 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:31:24.949631 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0181889-714e-432a-ab06-4950ed72b509 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:31:24.968538 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1190/2409] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:31:24.975194 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c8835-d6db-467c-a0d9-16438f92e148 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:31:25.008218 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c8835-d6db-467c-a0d9-16438f92e148 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:31:25.017560 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd309108-6c43-476a-9bf5-8c24caf17d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:31:25.018529 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd309108-6c43-476a-9bf5-8c24caf17d95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:31:25.019039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd309108-6c43-476a-9bf5-8c24caf17d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:31:25.019544 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd309108-6c43-476a-9bf5-8c24caf17d95 admin admin] PATH_INFO: `/v3/projects/0e44d69624be4720b0151f6cd16cde34/users/d29d4b602518487cb496bb26c6bf9780/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:31:25.024003 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd309108-6c43-476a-9bf5-8c24caf17d95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e44d69624be4720b0151f6cd16cde34, user_id=d29d4b602518487cb496bb26c6bf9780, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:31:25.029905 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd309108-6c43-476a-9bf5-8c24caf17d95 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:31:25.046903 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1220/2410] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:31:24 2022] PUT /identity/v3/projects/0e44d69624be4720b0151f6cd16cde34/users/d29d4b602518487cb496bb26c6bf9780/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:31:32.127148 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e70add-596b-41eb-89f2-6fde6cd6cd14 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:31:32.127764 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e70add-596b-41eb-89f2-6fde6cd6cd14 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:31:32.127764 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e70add-596b-41eb-89f2-6fde6cd6cd14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:31:32.161143 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-90e70add-596b-41eb-89f2-6fde6cd6cd14 None None] MFA Rules not processed for user `d29d4b602518487cb496bb26c6bf9780`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:31:32.164859 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-90e70add-596b-41eb-89f2-6fde6cd6cd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:31:32.260188 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:31:32.293267 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1191/2411] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:31:32 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:31:32.311998 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd309108-6c43-476a-9bf5-8c24caf17d95 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:31:32.314313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-fd309108-6c43-476a-9bf5-8c24caf17d95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:31:32.407015 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:31:32.413837 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd309108-6c43-476a-9bf5-8c24caf17d95 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:31:32.417926 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373f27b8-8c92-457a-89cf-2f2ea6d1575f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1193330cb5644765a7abb77738929ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:31:32.418599 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f27b8-8c92-457a-89cf-2f2ea6d1575f service glance] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:31:32.418887 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f27b8-8c92-457a-89cf-2f2ea6d1575f service glance] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:31:32.419169 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f27b8-8c92-457a-89cf-2f2ea6d1575f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:31:32.484749 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373f27b8-8c92-457a-89cf-2f2ea6d1575f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:31:32.487401 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373f27b8-8c92-457a-89cf-2f2ea6d1575f service glance] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:31:32.496998 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1221/2412] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:31:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:31:32.835288 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8e79f-98b8-4c4b-99b6-72269a777ea9 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:31:32.835288 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8e79f-98b8-4c4b-99b6-72269a777ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:31:32.835288 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8e79f-98b8-4c4b-99b6-72269a777ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:31:32.853989 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-4bc8e79f-98b8-4c4b-99b6-72269a777ea9 None None] MFA Rules not processed for user `b84dde57ff8e47419866377a819bea21`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:31:32.855247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4bc8e79f-98b8-4c4b-99b6-72269a777ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:31:32.937273 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1192/2413] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Jul 18 06:31:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:31:32.955742 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373f27b8-8c92-457a-89cf-2f2ea6d1575f service glance] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:31:32.957490 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-373f27b8-8c92-457a-89cf-2f2ea6d1575f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:31:33.056284 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373f27b8-8c92-457a-89cf-2f2ea6d1575f service glance] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:31:33.062087 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9836e42c-ef0c-4e6f-8044-eadde7304d40 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0681f5fbea6d415c9ad2f3afae256a5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:31:33.062607 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836e42c-ef0c-4e6f-8044-eadde7304d40 service swift] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:31:33.062914 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836e42c-ef0c-4e6f-8044-eadde7304d40 service swift] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:31:33.063180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836e42c-ef0c-4e6f-8044-eadde7304d40 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:31:33.073331 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9836e42c-ef0c-4e6f-8044-eadde7304d40 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:31:33.074982 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9836e42c-ef0c-4e6f-8044-eadde7304d40 service swift] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:31:33.084699 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1222/2414] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Jul 18 06:31:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 06:31:35.870287 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc8e79f-98b8-4c4b-99b6-72269a777ea9 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:31:35.872148 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4bc8e79f-98b8-4c4b-99b6-72269a777ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:31:36.034734 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc8e79f-98b8-4c4b-99b6-72269a777ea9 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:31:36.046410 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e777bb-b8ef-42be-94fb-6be192e0f5be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:31:36.048557 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e777bb-b8ef-42be-94fb-6be192e0f5be service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:31:36.048557 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e777bb-b8ef-42be-94fb-6be192e0f5be service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:31:36.048557 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e777bb-b8ef-42be-94fb-6be192e0f5be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:31:36.050459 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-13e777bb-b8ef-42be-94fb-6be192e0f5be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:31:36.139236 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e777bb-b8ef-42be-94fb-6be192e0f5be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:31:36.141051 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e777bb-b8ef-42be-94fb-6be192e0f5be service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:31:36.153532 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1193/2415] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:31:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:31:43.081190 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=59 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:31:46.012831 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae8487-1d41-4f72-8786-b662ecdb963a None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:31:46.013711 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae8487-1d41-4f72-8786-b662ecdb963a None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:31:46.013711 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae8487-1d41-4f72-8786-b662ecdb963a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:31:46.045189 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-2aae8487-1d41-4f72-8786-b662ecdb963a None None] MFA Rules not processed for user `81d9b70897ad4f8fabe72d1728558181`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:31:46.047469 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-2aae8487-1d41-4f72-8786-b662ecdb963a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:31:46.060299 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1223/2416] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:31:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:31:46.149216 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=85 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:31:46.462931 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e777bb-b8ef-42be-94fb-6be192e0f5be service nova] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:31:46.562803 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e777bb-b8ef-42be-94fb-6be192e0f5be service nova] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:31:46.568220 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44dd1d3-1bfd-4194-aa90-4ab2f5a821f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:31:46.569164 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44dd1d3-1bfd-4194-aa90-4ab2f5a821f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:31:46.569571 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44dd1d3-1bfd-4194-aa90-4ab2f5a821f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:31:46.569952 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44dd1d3-1bfd-4194-aa90-4ab2f5a821f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:31:46.572465 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a44dd1d3-1bfd-4194-aa90-4ab2f5a821f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:31:46.585997 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44dd1d3-1bfd-4194-aa90-4ab2f5a821f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:31:46.587896 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44dd1d3-1bfd-4194-aa90-4ab2f5a821f1 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:31:46.602343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1194/2417] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:31:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 06:31:46.757515 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae8487-1d41-4f72-8786-b662ecdb963a None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:31:46.771517 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae8487-1d41-4f72-8786-b662ecdb963a None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:31:46.777574 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4d66f6-66d6-4ce5-8131-a604b6140d94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:31:46.778543 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d66f6-66d6-4ce5-8131-a604b6140d94 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:31:46.778942 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d66f6-66d6-4ce5-8131-a604b6140d94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:31:46.779559 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d66f6-66d6-4ce5-8131-a604b6140d94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:31:46.782604 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-be4d66f6-66d6-4ce5-8131-a604b6140d94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:31:46.799782 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4d66f6-66d6-4ce5-8131-a604b6140d94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:31:46.802093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4d66f6-66d6-4ce5-8131-a604b6140d94 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:31:46.812864 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1224/2418] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:31:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 06:31:47.564549 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44dd1d3-1bfd-4194-aa90-4ab2f5a821f1 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:31:47.567333 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a44dd1d3-1bfd-4194-aa90-4ab2f5a821f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:31:47.675914 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44dd1d3-1bfd-4194-aa90-4ab2f5a821f1 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:31:47.682978 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb9f87-0874-4fdd-9c42-1f67c28859fa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3560534427544cdabfd0ba702a0a165', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:31:47.683652 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb9f87-0874-4fdd-9c42-1f67c28859fa service placement] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:31:47.683936 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb9f87-0874-4fdd-9c42-1f67c28859fa service placement] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:31:47.684211 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb9f87-0874-4fdd-9c42-1f67c28859fa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:31:47.686198 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f9fb9f87-0874-4fdd-9c42-1f67c28859fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:31:47.699185 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fb9f87-0874-4fdd-9c42-1f67c28859fa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:31:47.700849 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fb9f87-0874-4fdd-9c42-1f67c28859fa service placement] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:31:47.715102 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1195/2419] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:31:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:31:48.404458 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd43b5-2439-479a-81e0-29b6a69dab74 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:31:48.405945 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd43b5-2439-479a-81e0-29b6a69dab74 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:31:48.406569 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd43b5-2439-479a-81e0-29b6a69dab74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:31:48.444256 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-b1dd43b5-2439-479a-81e0-29b6a69dab74 None None] MFA Rules not processed for user `b84dde57ff8e47419866377a819bea21`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:31:48.446560 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-b1dd43b5-2439-479a-81e0-29b6a69dab74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:31:48.460252 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1225/2420] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Jul 18 06:31:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:31:48.478772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb9f87-0874-4fdd-9c42-1f67c28859fa service placement] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:31:48.499528 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb9f87-0874-4fdd-9c42-1f67c28859fa service placement] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:31:48.507644 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3b255a-a66a-4610-9ed2-6a4d3a35a8ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0681f5fbea6d415c9ad2f3afae256a5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:31:48.508312 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b255a-a66a-4610-9ed2-6a4d3a35a8ef service swift] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:31:48.508630 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b255a-a66a-4610-9ed2-6a4d3a35a8ef service swift] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:31:48.508909 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b255a-a66a-4610-9ed2-6a4d3a35a8ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:31:48.518209 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3b255a-a66a-4610-9ed2-6a4d3a35a8ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:31:48.519967 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3b255a-a66a-4610-9ed2-6a4d3a35a8ef service swift] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:31:48.530455 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1196/2421] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Jul 18 06:31:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 06:31:51.455144 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f7e6bb-4b3c-4b47-a486-6a75775cccc2 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:31:51.456033 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f7e6bb-4b3c-4b47-a486-6a75775cccc2 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:31:51.456033 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f7e6bb-4b3c-4b47-a486-6a75775cccc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:31:51.497731 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-87f7e6bb-4b3c-4b47-a486-6a75775cccc2 None None] MFA Rules not processed for user `b84dde57ff8e47419866377a819bea21`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:31:51.497731 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-87f7e6bb-4b3c-4b47-a486-6a75775cccc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:31:51.502714 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1226/2422] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Jul 18 06:31:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:31:51.525130 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3b255a-a66a-4610-9ed2-6a4d3a35a8ef service swift] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:31:51.542974 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3b255a-a66a-4610-9ed2-6a4d3a35a8ef service swift] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:31:51.566482 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f05df-a30b-43fe-81cb-86da65eda200 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0681f5fbea6d415c9ad2f3afae256a5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:31:51.568469 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f05df-a30b-43fe-81cb-86da65eda200 service swift] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:31:51.569554 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f05df-a30b-43fe-81cb-86da65eda200 service swift] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:31:51.570553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f05df-a30b-43fe-81cb-86da65eda200 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:31:51.590917 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776f05df-a30b-43fe-81cb-86da65eda200 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:31:51.595118 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776f05df-a30b-43fe-81cb-86da65eda200 service swift] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:31:51.614094 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1197/2423] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Jul 18 06:31:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 06:32:00.372224 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f7e6bb-4b3c-4b47-a486-6a75775cccc2 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:32:00.389484 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f7e6bb-4b3c-4b47-a486-6a75775cccc2 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:32:00.398281 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4295f-cad0-4748-9aa6-638e0d32607f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:32:00.398971 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4295f-cad0-4748-9aa6-638e0d32607f service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:32:00.399290 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4295f-cad0-4748-9aa6-638e0d32607f service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:32:00.399641 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4295f-cad0-4748-9aa6-638e0d32607f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:32:00.402022 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-57e4295f-cad0-4748-9aa6-638e0d32607f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:32:00.417015 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4295f-cad0-4748-9aa6-638e0d32607f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:32:00.418674 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4295f-cad0-4748-9aa6-638e0d32607f service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:32:00.433713 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1227/2424] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:32:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:32:01.606572 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=70 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:32:10.426250 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=21 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:32:15.061682 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33095ed-49b0-47e2-9eeb-37a1906aba90 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:32:15.063193 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33095ed-49b0-47e2-9eeb-37a1906aba90 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:32:15.063624 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33095ed-49b0-47e2-9eeb-37a1906aba90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:32:15.087943 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-c33095ed-49b0-47e2-9eeb-37a1906aba90 None None] MFA Rules not processed for user `76f3dd56c5d24c7490e337b2aa90b584`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:32:15.089270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-c33095ed-49b0-47e2-9eeb-37a1906aba90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:32:15.098130 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1198/2425] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:32:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:32:16.057838 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4295f-cad0-4748-9aa6-638e0d32607f service nova] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:32:16.061571 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-57e4295f-cad0-4748-9aa6-638e0d32607f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:32:16.080873 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4295f-cad0-4748-9aa6-638e0d32607f service nova] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:32:16.087909 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a0971-9312-40e2-b3e7-b8042f59dff9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:32:16.088860 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a0971-9312-40e2-b3e7-b8042f59dff9 service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:32:16.089263 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a0971-9312-40e2-b3e7-b8042f59dff9 service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:32:16.089648 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a0971-9312-40e2-b3e7-b8042f59dff9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:32:16.092063 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-9f6a0971-9312-40e2-b3e7-b8042f59dff9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:32:16.107134 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6a0971-9312-40e2-b3e7-b8042f59dff9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:32:16.109104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6a0971-9312-40e2-b3e7-b8042f59dff9 service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:32:16.122690 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1228/2426] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:32:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:32:20.528007 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33095ed-49b0-47e2-9eeb-37a1906aba90 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:32:20.547111 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33095ed-49b0-47e2-9eeb-37a1906aba90 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:32:20.553283 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d4287-99fe-4911-9ed0-48910e7c2677 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:32:20.554805 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d4287-99fe-4911-9ed0-48910e7c2677 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:32:20.555578 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d4287-99fe-4911-9ed0-48910e7c2677 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:32:20.555578 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d4287-99fe-4911-9ed0-48910e7c2677 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:32:20.559461 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-c05d4287-99fe-4911-9ed0-48910e7c2677 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:32:20.578933 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d4287-99fe-4911-9ed0-48910e7c2677 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:32:20.582320 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d4287-99fe-4911-9ed0-48910e7c2677 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:32:20.595271 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1199/2427] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:32:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:32:20.983530 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b8ae7-1e6e-408a-b86f-5245aedb8aa1 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:32:20.984146 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b8ae7-1e6e-408a-b86f-5245aedb8aa1 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:32:20.984146 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b8ae7-1e6e-408a-b86f-5245aedb8aa1 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:32:20.986061 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1229/2428] 10.222.0.40 () {54 vars in 921 bytes} [Mon Jul 18 06:32:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 06:32:20.993923 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8708475-a558-46d3-986f-6c0caf639096 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:32:20.994345 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8708475-a558-46d3-986f-6c0caf639096 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:32:20.994718 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8708475-a558-46d3-986f-6c0caf639096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:32:21.014969 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-a8708475-a558-46d3-986f-6c0caf639096 None None] MFA Rules not processed for user `f92d948810ea4241810473506b654f7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:32:21.016192 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a8708475-a558-46d3-986f-6c0caf639096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:32:21.026404 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1200/2429] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Jul 18 06:32:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:32:21.038936 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8b8ae7-1e6e-408a-b86f-5245aedb8aa1 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:32:21.041277 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-3e8b8ae7-1e6e-408a-b86f-5245aedb8aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:32:21.058505 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8b8ae7-1e6e-408a-b86f-5245aedb8aa1 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:32:21.063595 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8729a381-dd11-4039-821a-5958b2d38fa0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1193330cb5644765a7abb77738929ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:32:21.064874 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729a381-dd11-4039-821a-5958b2d38fa0 service glance] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:32:21.065338 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729a381-dd11-4039-821a-5958b2d38fa0 service glance] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:32:21.065775 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729a381-dd11-4039-821a-5958b2d38fa0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:32:21.080219 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8729a381-dd11-4039-821a-5958b2d38fa0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:32:21.082210 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8729a381-dd11-4039-821a-5958b2d38fa0 service glance] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:32:21.096408 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1230/2430] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:32:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 06:32:21.303600 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea8c41-9f7f-49dd-9412-b88119e43311 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:32:21.303887 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea8c41-9f7f-49dd-9412-b88119e43311 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:32:21.304166 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea8c41-9f7f-49dd-9412-b88119e43311 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:32:21.305680 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1201/2431] 10.222.0.40 () {54 vars in 921 bytes} [Mon Jul 18 06:32:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:32:21.311418 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710a040-8949-46e3-ab11-5bea22712df6 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:32:21.311833 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710a040-8949-46e3-ab11-5bea22712df6 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:32:21.311993 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710a040-8949-46e3-ab11-5bea22712df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:32:21.330866 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-6710a040-8949-46e3-ab11-5bea22712df6 None None] MFA Rules not processed for user `9824a96e2bcd4a16bfc6fe6bf4ea1ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:32:21.332148 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-6710a040-8949-46e3-ab11-5bea22712df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:32:21.339109 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1231/2432] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Jul 18 06:32:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:32:21.353779 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea8c41-9f7f-49dd-9412-b88119e43311 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:32:21.356190 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-acea8c41-9f7f-49dd-9412-b88119e43311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:32:21.373888 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea8c41-9f7f-49dd-9412-b88119e43311 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:32:21.377769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3db61c-6dd5-425a-b6ba-f8ecd44bace0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:32:21.378637 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3db61c-6dd5-425a-b6ba-f8ecd44bace0 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:32:21.378933 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3db61c-6dd5-425a-b6ba-f8ecd44bace0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:32:21.379247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3db61c-6dd5-425a-b6ba-f8ecd44bace0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:32:21.388243 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3db61c-6dd5-425a-b6ba-f8ecd44bace0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:32:21.389802 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3db61c-6dd5-425a-b6ba-f8ecd44bace0 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:32:21.399914 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1202/2433] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:32:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:32:21.505801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c7ecf-4da7-428c-a151-6c61a76a4a42 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:32:21.506191 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c7ecf-4da7-428c-a151-6c61a76a4a42 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:32:21.506480 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c7ecf-4da7-428c-a151-6c61a76a4a42 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:32:21.508038 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1232/2434] 10.222.0.40 () {54 vars in 921 bytes} [Mon Jul 18 06:32:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 06:32:21.514875 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2773a11-8844-48c4-b5de-9ca5a84d230f None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:32:21.515301 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2773a11-8844-48c4-b5de-9ca5a84d230f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:32:21.515655 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2773a11-8844-48c4-b5de-9ca5a84d230f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:32:21.537320 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-e2773a11-8844-48c4-b5de-9ca5a84d230f None None] MFA Rules not processed for user `f92d948810ea4241810473506b654f7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:32:21.538595 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e2773a11-8844-48c4-b5de-9ca5a84d230f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:32:21.548449 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1203/2435] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Jul 18 06:32:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:32:21.560165 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220c7ecf-4da7-428c-a151-6c61a76a4a42 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:32:21.576538 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220c7ecf-4da7-428c-a151-6c61a76a4a42 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:32:21.580394 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe581c8d-cb6f-487b-bba5-ca0fceefc397 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:32:21.581658 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe581c8d-cb6f-487b-bba5-ca0fceefc397 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:32:21.581952 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe581c8d-cb6f-487b-bba5-ca0fceefc397 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:32:21.582228 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe581c8d-cb6f-487b-bba5-ca0fceefc397 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:32:21.594750 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe581c8d-cb6f-487b-bba5-ca0fceefc397 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:32:21.596324 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe581c8d-cb6f-487b-bba5-ca0fceefc397 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:32:21.610054 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1233/2436] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:32:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:32:24.942213 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2773a11-8844-48c4-b5de-9ca5a84d230f None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:32:24.970215 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2773a11-8844-48c4-b5de-9ca5a84d230f None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:32:24.982345 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2652de-471d-4641-abaa-bdefc2b23181 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:32:24.984602 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2652de-471d-4641-abaa-bdefc2b23181 service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:32:24.985572 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2652de-471d-4641-abaa-bdefc2b23181 service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:32:24.990600 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2652de-471d-4641-abaa-bdefc2b23181 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:32:24.990600 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-dc2652de-471d-4641-abaa-bdefc2b23181 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:32:25.117557 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2652de-471d-4641-abaa-bdefc2b23181 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:32:25.121257 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2652de-471d-4641-abaa-bdefc2b23181 service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:32:25.155270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1204/2437] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:32:24 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:32:27.850528 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc5013-97f7-4cd2-a5cf-21ea3bc1c84e None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:32:27.851316 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc5013-97f7-4cd2-a5cf-21ea3bc1c84e None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:32:27.852271 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc5013-97f7-4cd2-a5cf-21ea3bc1c84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:32:27.895516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-debc5013-97f7-4cd2-a5cf-21ea3bc1c84e None None] MFA Rules not processed for user `b84dde57ff8e47419866377a819bea21`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:32:27.897896 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-debc5013-97f7-4cd2-a5cf-21ea3bc1c84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:32:27.914432 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1234/2438] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Jul 18 06:32:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:32:27.939554 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2652de-471d-4641-abaa-bdefc2b23181 service nova] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:32:27.955624 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2652de-471d-4641-abaa-bdefc2b23181 service nova] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:32:27.964242 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f80ff1-bd69-4587-b9c5-80d153bc039b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0681f5fbea6d415c9ad2f3afae256a5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:32:27.966094 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f80ff1-bd69-4587-b9c5-80d153bc039b service swift] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:32:27.966982 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f80ff1-bd69-4587-b9c5-80d153bc039b service swift] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:32:27.967889 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f80ff1-bd69-4587-b9c5-80d153bc039b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:32:27.986600 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f80ff1-bd69-4587-b9c5-80d153bc039b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:32:27.990647 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f80ff1-bd69-4587-b9c5-80d153bc039b service swift] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:32:28.021985 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1205/2439] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Jul 18 06:32:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 06:32:37.886280 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=24 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:32:38.000968 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=55 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:32:47.484380 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc5013-97f7-4cd2-a5cf-21ea3bc1c84e None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:32:47.494644 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc5013-97f7-4cd2-a5cf-21ea3bc1c84e None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:32:47.498273 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f44697f-2f8b-45c9-bf72-552cfa6f1644 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:32:47.498951 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f44697f-2f8b-45c9-bf72-552cfa6f1644 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:32:47.499243 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f44697f-2f8b-45c9-bf72-552cfa6f1644 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:32:47.499528 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f44697f-2f8b-45c9-bf72-552cfa6f1644 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:32:47.510623 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f44697f-2f8b-45c9-bf72-552cfa6f1644 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:32:47.512077 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f44697f-2f8b-45c9-bf72-552cfa6f1644 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:32:47.526127 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1235/2440] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:32:47 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:32:51.780776 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e03ff63-b01e-4836-a9c7-58654febe407 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:32:51.781339 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e03ff63-b01e-4836-a9c7-58654febe407 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:32:51.781462 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e03ff63-b01e-4836-a9c7-58654febe407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:32:51.801398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-5e03ff63-b01e-4836-a9c7-58654febe407 None None] MFA Rules not processed for user `b84dde57ff8e47419866377a819bea21`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:32:51.802659 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-5e03ff63-b01e-4836-a9c7-58654febe407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:32:51.812533 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1206/2441] 10.222.0.40 () {58 vars in 967 bytes} [Mon Jul 18 06:32:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:32:51.826324 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f44697f-2f8b-45c9-bf72-552cfa6f1644 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:32:51.839462 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f44697f-2f8b-45c9-bf72-552cfa6f1644 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:32:51.849606 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa1e156-6b0a-4467-aa56-525f66f40648 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0681f5fbea6d415c9ad2f3afae256a5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:32:51.850393 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1e156-6b0a-4467-aa56-525f66f40648 service swift] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:32:51.850883 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1e156-6b0a-4467-aa56-525f66f40648 service swift] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:32:51.851194 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1e156-6b0a-4467-aa56-525f66f40648 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:32:51.861031 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa1e156-6b0a-4467-aa56-525f66f40648 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:32:51.865898 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa1e156-6b0a-4467-aa56-525f66f40648 service swift] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:32:51.873446 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1236/2442] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Jul 18 06:32:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 06:33:01.794852 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:33:01.869722 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=12 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:33:22.820959 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a305c1-8991-40ce-8ce7-938236217b1d None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:33:22.821558 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a305c1-8991-40ce-8ce7-938236217b1d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:33:22.821558 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a305c1-8991-40ce-8ce7-938236217b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:33:22.840233 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-e0a305c1-8991-40ce-8ce7-938236217b1d None None] MFA Rules not processed for user `621a7ea4f60545d5a5c30b2b7a370bf0`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:33:22.841510 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e0a305c1-8991-40ce-8ce7-938236217b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:33:22.899209 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1207/2443] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:33:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:33:32.892080 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=25 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:33:47.947121 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa1e156-6b0a-4467-aa56-525f66f40648 service swift] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:33:47.960465 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa1e156-6b0a-4467-aa56-525f66f40648 service swift] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:33:47.967185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869adf4-6ae5-400f-b9b3-89e1b9137abb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:33:47.967784 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869adf4-6ae5-400f-b9b3-89e1b9137abb service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:33:47.968133 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869adf4-6ae5-400f-b9b3-89e1b9137abb service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:33:47.968444 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869adf4-6ae5-400f-b9b3-89e1b9137abb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:33:47.970501 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-8869adf4-6ae5-400f-b9b3-89e1b9137abb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:33:47.984506 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8869adf4-6ae5-400f-b9b3-89e1b9137abb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:33:47.986049 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8869adf4-6ae5-400f-b9b3-89e1b9137abb service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:33:47.998180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1237/2444] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:33:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:33:55.682372 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a305c1-8991-40ce-8ce7-938236217b1d None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:33:55.692883 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a305c1-8991-40ce-8ce7-938236217b1d None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:33:55.696476 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7892fa6-d33d-4d09-8b2a-e3cae7d8691e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:33:55.697124 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7892fa6-d33d-4d09-8b2a-e3cae7d8691e service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:33:55.697408 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7892fa6-d33d-4d09-8b2a-e3cae7d8691e service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:33:55.697684 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7892fa6-d33d-4d09-8b2a-e3cae7d8691e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:33:55.700150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-c7892fa6-d33d-4d09-8b2a-e3cae7d8691e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:33:55.764906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7892fa6-d33d-4d09-8b2a-e3cae7d8691e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:33:55.766296 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7892fa6-d33d-4d09-8b2a-e3cae7d8691e service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:33:55.776092 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1208/2445] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:33:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 06:33:57.992722 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=6 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:33:59.585466 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869adf4-6ae5-400f-b9b3-89e1b9137abb service nova] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:33:59.653588 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869adf4-6ae5-400f-b9b3-89e1b9137abb service nova] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:33:59.659749 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61eca3-b967-484c-82e1-96eee7221a66 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3560534427544cdabfd0ba702a0a165', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:33:59.660429 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61eca3-b967-484c-82e1-96eee7221a66 service placement] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:33:59.660737 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61eca3-b967-484c-82e1-96eee7221a66 service placement] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:33:59.661022 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61eca3-b967-484c-82e1-96eee7221a66 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:33:59.662952 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-5b61eca3-b967-484c-82e1-96eee7221a66 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:33:59.701435 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b61eca3-b967-484c-82e1-96eee7221a66 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:33:59.703074 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b61eca3-b967-484c-82e1-96eee7221a66 service placement] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:33:59.716901 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1238/2446] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:33:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:33:59.725523 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7892fa6-d33d-4d09-8b2a-e3cae7d8691e service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:33:59.727065 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-c7892fa6-d33d-4d09-8b2a-e3cae7d8691e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:33:59.813036 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7892fa6-d33d-4d09-8b2a-e3cae7d8691e service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:33:59.817555 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dbe7d0-9db5-484e-adb7-21529a571d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:33:59.818442 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbe7d0-9db5-484e-adb7-21529a571d8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:33:59.818736 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbe7d0-9db5-484e-adb7-21529a571d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:33:59.819018 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbe7d0-9db5-484e-adb7-21529a571d8b admin admin] PATH_INFO: `/v3/users/76f3dd56c5d24c7490e337b2aa90b584` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:33:59.820244 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dbe7d0-9db5-484e-adb7-21529a571d8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=76f3dd56c5d24c7490e337b2aa90b584)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:33:59.822235 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dbe7d0-9db5-484e-adb7-21529a571d8b admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:00.037220 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b1dbe7d0-9db5-484e-adb7-21529a571d8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76f3dd56c5d24c7490e337b2aa90b584'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:00.046227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b1dbe7d0-9db5-484e-adb7-21529a571d8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76f3dd56c5d24c7490e337b2aa90b584'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:00.060300 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b1dbe7d0-9db5-484e-adb7-21529a571d8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76f3dd56c5d24c7490e337b2aa90b584'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:00.061121 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b1dbe7d0-9db5-484e-adb7-21529a571d8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76f3dd56c5d24c7490e337b2aa90b584'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:00.062880 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b1dbe7d0-9db5-484e-adb7-21529a571d8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76f3dd56c5d24c7490e337b2aa90b584'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:00.153008 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1209/2447] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:33:59 2022] DELETE /identity/v3/users/76f3dd56c5d24c7490e337b2aa90b584 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:34:00.413263 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61eca3-b967-484c-82e1-96eee7221a66 service placement] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:00.425112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61eca3-b967-484c-82e1-96eee7221a66 service placement] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:00.499487 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d001023-72ed-4909-aee7-d3f2789f22be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:00.500435 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d001023-72ed-4909-aee7-d3f2789f22be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:00.500784 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d001023-72ed-4909-aee7-d3f2789f22be admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:00.501160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d001023-72ed-4909-aee7-d3f2789f22be admin admin] PATH_INFO: `/v3/projects/c8f4f0c21b95422f80245b76b56367fb` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:00.502662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d001023-72ed-4909-aee7-d3f2789f22be admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8f4f0c21b95422f80245b76b56367fb)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:00.505305 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d001023-72ed-4909-aee7-d3f2789f22be admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:00.660299 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1d001023-72ed-4909-aee7-d3f2789f22be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8f4f0c21b95422f80245b76b56367fb', 'request_id': 'req-1d001023-72ed-4909-aee7-d3f2789f22be'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:00.667509 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1d001023-72ed-4909-aee7-d3f2789f22be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8f4f0c21b95422f80245b76b56367fb', 'request_id': 'req-1d001023-72ed-4909-aee7-d3f2789f22be'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:00.747538 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1d001023-72ed-4909-aee7-d3f2789f22be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8f4f0c21b95422f80245b76b56367fb', 'request_id': 'req-1d001023-72ed-4909-aee7-d3f2789f22be'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:00.761250 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1d001023-72ed-4909-aee7-d3f2789f22be admin admin] The token cache is being invalidate because project c8f4f0c21b95422f80245b76b56367fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:34:00.761611 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1d001023-72ed-4909-aee7-d3f2789f22be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:00.762801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1d001023-72ed-4909-aee7-d3f2789f22be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:00.764678 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1239/2448] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:34:00 2022] DELETE /identity/v3/projects/c8f4f0c21b95422f80245b76b56367fb => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:34:00.769998 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a5e15-e8d2-4b81-ab4a-c19787a749ad None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:00.770597 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a5e15-e8d2-4b81-ab4a-c19787a749ad None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:00.770597 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a5e15-e8d2-4b81-ab4a-c19787a749ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:00.775869 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:34:00.836049 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-e62a5e15-e8d2-4b81-ab4a-c19787a749ad None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:34:00.837332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e62a5e15-e8d2-4b81-ab4a-c19787a749ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:00.913612 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1210/2449] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:34:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:34:00.922312 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbbabbc-c2fe-4531-8ed7-5caff313d88b None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:00.922563 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbbabbc-c2fe-4531-8ed7-5caff313d88b None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:00.922799 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbbabbc-c2fe-4531-8ed7-5caff313d88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:00.942477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-acbbabbc-c2fe-4531-8ed7-5caff313d88b None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:34:00.943739 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-acbbabbc-c2fe-4531-8ed7-5caff313d88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:00.953091 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1240/2450] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:34:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:34:00.957646 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a5e15-e8d2-4b81-ab4a-c19787a749ad None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:00.969210 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a5e15-e8d2-4b81-ab4a-c19787a749ad None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:00.974411 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6860a5f-1f2e-4d84-b224-955b3510152d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:00.975127 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6860a5f-1f2e-4d84-b224-955b3510152d admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:00.975418 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6860a5f-1f2e-4d84-b224-955b3510152d admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:00.975737 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6860a5f-1f2e-4d84-b224-955b3510152d admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:00.976592 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6860a5f-1f2e-4d84-b224-955b3510152d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:34:00.976907 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6860a5f-1f2e-4d84-b224-955b3510152d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:00.978714 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6860a5f-1f2e-4d84-b224-955b3510152d admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:00.999645 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1211/2451] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:34:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:34:01.006684 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbbabbc-c2fe-4531-8ed7-5caff313d88b None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:01.016842 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:34:01.021474 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbbabbc-c2fe-4531-8ed7-5caff313d88b None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:01.027211 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b69f4f9-1bae-4885-9eec-f84d2ba0ee4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:01.028912 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69f4f9-1bae-4885-9eec-f84d2ba0ee4c admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:01.029429 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69f4f9-1bae-4885-9eec-f84d2ba0ee4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:01.029772 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69f4f9-1bae-4885-9eec-f84d2ba0ee4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:01.030577 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b69f4f9-1bae-4885-9eec-f84d2ba0ee4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:01.032962 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b69f4f9-1bae-4885-9eec-f84d2ba0ee4c admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:01.200147 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1241/2452] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:34:01 2022] POST /identity/v3/projects => generated 369 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:34:01.204724 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6860a5f-1f2e-4d84-b224-955b3510152d admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:01.218332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6860a5f-1f2e-4d84-b224-955b3510152d admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:01.256014 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b69f4f9-1bae-4885-9eec-f84d2ba0ee4c admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:01.258194 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-9b69f4f9-1bae-4885-9eec-f84d2ba0ee4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:01.301534 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946fea36-ece8-47ab-9bb4-deab7125b934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:01.302511 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946fea36-ece8-47ab-9bb4-deab7125b934 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:01.302836 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946fea36-ece8-47ab-9bb4-deab7125b934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:01.303183 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946fea36-ece8-47ab-9bb4-deab7125b934 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:01.304133 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946fea36-ece8-47ab-9bb4-deab7125b934 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:01.306643 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946fea36-ece8-47ab-9bb4-deab7125b934 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:01.385917 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b69f4f9-1bae-4885-9eec-f84d2ba0ee4c admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:01.389400 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1212/2453] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:34:01 2022] POST /identity/v3/users => generated 285 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:34:01.393911 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946fea36-ece8-47ab-9bb4-deab7125b934 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:01.395341 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c3460-2251-4598-8c57-af30c7c7387f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:01.396096 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c3460-2251-4598-8c57-af30c7c7387f service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:01.396386 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c3460-2251-4598-8c57-af30c7c7387f service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:01.396871 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c3460-2251-4598-8c57-af30c7c7387f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:01.398946 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-306c3460-2251-4598-8c57-af30c7c7387f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:01.406966 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946fea36-ece8-47ab-9bb4-deab7125b934 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:01.412997 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52da73-173e-4d9d-afdc-084849873dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:01.413789 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52da73-173e-4d9d-afdc-084849873dfe admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:01.414179 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52da73-173e-4d9d-afdc-084849873dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:01.414566 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52da73-173e-4d9d-afdc-084849873dfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:01.415469 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e52da73-173e-4d9d-afdc-084849873dfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:01.417484 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e52da73-173e-4d9d-afdc-084849873dfe admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:01.434019 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1213/2454] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:34:01 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 06:34:01.439313 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52da73-173e-4d9d-afdc-084849873dfe admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:01.450782 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52da73-173e-4d9d-afdc-084849873dfe admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:01.455542 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3aec3b-997e-4fa2-8b3a-21c1cff29d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:01.456216 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3aec3b-997e-4fa2-8b3a-21c1cff29d7b admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:01.456535 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3aec3b-997e-4fa2-8b3a-21c1cff29d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:01.456816 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3aec3b-997e-4fa2-8b3a-21c1cff29d7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:01.457565 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3aec3b-997e-4fa2-8b3a-21c1cff29d7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:01.459165 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3aec3b-997e-4fa2-8b3a-21c1cff29d7b admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:01.472520 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1214/2455] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:34:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:34:01.476764 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3aec3b-997e-4fa2-8b3a-21c1cff29d7b admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:01.480837 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c3460-2251-4598-8c57-af30c7c7387f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:01.482870 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c3460-2251-4598-8c57-af30c7c7387f service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:01.489974 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3aec3b-997e-4fa2-8b3a-21c1cff29d7b admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:01.495026 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1242/2456] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:34:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:34:01.497110 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d22cb-be33-4a01-a2d6-84e1807503a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:01.498339 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d22cb-be33-4a01-a2d6-84e1807503a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:01.498710 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d22cb-be33-4a01-a2d6-84e1807503a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:01.498991 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d22cb-be33-4a01-a2d6-84e1807503a8 admin admin] PATH_INFO: `/v3/projects/f87ba55c18874e4e83ced262286d2d32/users/8975cf8682194891a07eda0b9f6bb87c/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:01.516987 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825d22cb-be33-4a01-a2d6-84e1807503a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f87ba55c18874e4e83ced262286d2d32, user_id=8975cf8682194891a07eda0b9f6bb87c, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:01.520653 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825d22cb-be33-4a01-a2d6-84e1807503a8 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:01.599372 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1215/2457] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:34:01 2022] PUT /identity/v3/projects/f87ba55c18874e4e83ced262286d2d32/users/8975cf8682194891a07eda0b9f6bb87c/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:34:01.618060 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7e455-2cac-47b2-b77f-7c8b875e7e23 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:01.618305 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7e455-2cac-47b2-b77f-7c8b875e7e23 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:01.618910 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7e455-2cac-47b2-b77f-7c8b875e7e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:01.676742 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-b5d7e455-2cac-47b2-b77f-7c8b875e7e23 None None] MFA Rules not processed for user `8975cf8682194891a07eda0b9f6bb87c`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:34:01.678706 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-b5d7e455-2cac-47b2-b77f-7c8b875e7e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:01.750166 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:34:01.781599 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1243/2458] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:34:01 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:34:01.786015 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d22cb-be33-4a01-a2d6-84e1807503a8 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:01.792199 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:34:01.878515 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d22cb-be33-4a01-a2d6-84e1807503a8 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:01.885264 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13d41e-bf3b-43f2-bcfe-a43e24d52f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:01.885942 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13d41e-bf3b-43f2-bcfe-a43e24d52f95 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:01.886232 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13d41e-bf3b-43f2-bcfe-a43e24d52f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:01.886519 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13d41e-bf3b-43f2-bcfe-a43e24d52f95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:01.887801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd13d41e-bf3b-43f2-bcfe-a43e24d52f95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:01.889818 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd13d41e-bf3b-43f2-bcfe-a43e24d52f95 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:01.991608 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1216/2459] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:34:01 2022] POST /identity/v3/projects => generated 367 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:34:01.996516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d7e455-2cac-47b2-b77f-7c8b875e7e23 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:02.010923 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d7e455-2cac-47b2-b77f-7c8b875e7e23 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:02.108962 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59afd28-f5fa-4b73-9a69-cee7ecf3ded2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:02.110392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59afd28-f5fa-4b73-9a69-cee7ecf3ded2 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:02.110392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59afd28-f5fa-4b73-9a69-cee7ecf3ded2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:02.110584 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59afd28-f5fa-4b73-9a69-cee7ecf3ded2 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:02.111886 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59afd28-f5fa-4b73-9a69-cee7ecf3ded2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:02.114506 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59afd28-f5fa-4b73-9a69-cee7ecf3ded2 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:02.231798 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1244/2460] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:34:01 2022] POST /identity/v3/users => generated 290 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:34:02.234469 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13d41e-bf3b-43f2-bcfe-a43e24d52f95 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:02.249257 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13d41e-bf3b-43f2-bcfe-a43e24d52f95 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:02.254337 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15016fd3-5f2b-40e6-8ab6-7763f1bf2bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:02.255025 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15016fd3-5f2b-40e6-8ab6-7763f1bf2bff admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:02.255312 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15016fd3-5f2b-40e6-8ab6-7763f1bf2bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:02.255623 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15016fd3-5f2b-40e6-8ab6-7763f1bf2bff admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:02.256365 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15016fd3-5f2b-40e6-8ab6-7763f1bf2bff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:02.258010 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15016fd3-5f2b-40e6-8ab6-7763f1bf2bff admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:02.271069 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1217/2461] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:34:02.275772 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59afd28-f5fa-4b73-9a69-cee7ecf3ded2 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:02.286612 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59afd28-f5fa-4b73-9a69-cee7ecf3ded2 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:02.292569 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e14cd5f-552a-469e-adf3-12cac1e56243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:02.293794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14cd5f-552a-469e-adf3-12cac1e56243 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:02.294218 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14cd5f-552a-469e-adf3-12cac1e56243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:02.294681 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14cd5f-552a-469e-adf3-12cac1e56243 admin admin] PATH_INFO: `/v3/projects/3272e915599e47afa1b5cc3f8ea604c4/users/be9e4b46b1d94f1183ddbdf8f7ac5c50/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:02.313910 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e14cd5f-552a-469e-adf3-12cac1e56243 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3272e915599e47afa1b5cc3f8ea604c4, user_id=be9e4b46b1d94f1183ddbdf8f7ac5c50, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:02.317425 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e14cd5f-552a-469e-adf3-12cac1e56243 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:02.394145 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1245/2462] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:34:02 2022] PUT /identity/v3/projects/3272e915599e47afa1b5cc3f8ea604c4/users/be9e4b46b1d94f1183ddbdf8f7ac5c50/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:34:02.404909 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15016fd3-5f2b-40e6-8ab6-7763f1bf2bff admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:02.415532 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15016fd3-5f2b-40e6-8ab6-7763f1bf2bff admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:02.477422 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc654f15-ec79-4f44-88ab-9a762d479727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:02.478108 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc654f15-ec79-4f44-88ab-9a762d479727 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:02.478397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc654f15-ec79-4f44-88ab-9a762d479727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:02.478677 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc654f15-ec79-4f44-88ab-9a762d479727 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:02.479413 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc654f15-ec79-4f44-88ab-9a762d479727 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:02.481949 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc654f15-ec79-4f44-88ab-9a762d479727 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:02.494550 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1218/2463] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:34:02.499823 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e14cd5f-552a-469e-adf3-12cac1e56243 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:02.512748 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e14cd5f-552a-469e-adf3-12cac1e56243 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:02.518528 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc8748-6ebd-4274-aaed-9f23268d2933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:02.519455 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc8748-6ebd-4274-aaed-9f23268d2933 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:02.519743 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc8748-6ebd-4274-aaed-9f23268d2933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:02.520131 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc8748-6ebd-4274-aaed-9f23268d2933 admin admin] PATH_INFO: `/v3/projects/3272e915599e47afa1b5cc3f8ea604c4/users/be9e4b46b1d94f1183ddbdf8f7ac5c50/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:02.522854 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc8748-6ebd-4274-aaed-9f23268d2933 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3272e915599e47afa1b5cc3f8ea604c4, user_id=be9e4b46b1d94f1183ddbdf8f7ac5c50, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:02.525806 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc8748-6ebd-4274-aaed-9f23268d2933 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:02.540758 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1246/2464] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:34:02 2022] PUT /identity/v3/projects/3272e915599e47afa1b5cc3f8ea604c4/users/be9e4b46b1d94f1183ddbdf8f7ac5c50/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:34:02.553498 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b067d28-a416-452f-bbd4-8e3ff0e87be5 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:02.553784 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b067d28-a416-452f-bbd4-8e3ff0e87be5 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:02.554064 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b067d28-a416-452f-bbd4-8e3ff0e87be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:02.587335 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-2b067d28-a416-452f-bbd4-8e3ff0e87be5 None None] MFA Rules not processed for user `be9e4b46b1d94f1183ddbdf8f7ac5c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:34:02.591276 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-2b067d28-a416-452f-bbd4-8e3ff0e87be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:02.670080 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:34:02.674685 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1219/2465] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:34:02 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:34:02.688984 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc8748-6ebd-4274-aaed-9f23268d2933 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:02.690719 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-3ebc8748-6ebd-4274-aaed-9f23268d2933 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:02.767668 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc8748-6ebd-4274-aaed-9f23268d2933 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:02.771373 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2333984d-98c1-4c2b-89fd-b056499b9f27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:02.772039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333984d-98c1-4c2b-89fd-b056499b9f27 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:02.772350 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333984d-98c1-4c2b-89fd-b056499b9f27 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:02.772663 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333984d-98c1-4c2b-89fd-b056499b9f27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:02.784172 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2333984d-98c1-4c2b-89fd-b056499b9f27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:02.789651 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2333984d-98c1-4c2b-89fd-b056499b9f27 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:02.797196 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1247/2466] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:34:02 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:34:06.251889 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b067d28-a416-452f-bbd4-8e3ff0e87be5 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:06.321474 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b067d28-a416-452f-bbd4-8e3ff0e87be5 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:06.326654 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da004523-fc4e-4a7e-8fba-fe7c7cd63fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:06.327535 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da004523-fc4e-4a7e-8fba-fe7c7cd63fb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:06.327819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da004523-fc4e-4a7e-8fba-fe7c7cd63fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:06.328098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da004523-fc4e-4a7e-8fba-fe7c7cd63fb8 admin admin] PATH_INFO: `/v3/users/8975cf8682194891a07eda0b9f6bb87c` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:06.329317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da004523-fc4e-4a7e-8fba-fe7c7cd63fb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8975cf8682194891a07eda0b9f6bb87c)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:06.331245 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da004523-fc4e-4a7e-8fba-fe7c7cd63fb8 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:06.760860 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-da004523-fc4e-4a7e-8fba-fe7c7cd63fb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8975cf8682194891a07eda0b9f6bb87c'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:06.769594 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-da004523-fc4e-4a7e-8fba-fe7c7cd63fb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8975cf8682194891a07eda0b9f6bb87c'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:06.785128 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-da004523-fc4e-4a7e-8fba-fe7c7cd63fb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8975cf8682194891a07eda0b9f6bb87c'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:06.785874 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-da004523-fc4e-4a7e-8fba-fe7c7cd63fb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8975cf8682194891a07eda0b9f6bb87c'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:06.786488 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-da004523-fc4e-4a7e-8fba-fe7c7cd63fb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8975cf8682194891a07eda0b9f6bb87c'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:06.876926 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1220/2467] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:34:06 2022] DELETE /identity/v3/users/8975cf8682194891a07eda0b9f6bb87c => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:34:06.881832 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00b007-3b05-4e85-a1ea-5836eac21a2d None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:06.882494 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00b007-3b05-4e85-a1ea-5836eac21a2d None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:06.882494 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00b007-3b05-4e85-a1ea-5836eac21a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:06.900895 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-6a00b007-3b05-4e85-a1ea-5836eac21a2d None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:34:06.902154 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-6a00b007-3b05-4e85-a1ea-5836eac21a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:06.956806 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:34:06.992538 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1248/2468] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:34:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:34:07.003214 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da004523-fc4e-4a7e-8fba-fe7c7cd63fb8 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:07.004735 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-da004523-fc4e-4a7e-8fba-fe7c7cd63fb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:07.072357 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da004523-fc4e-4a7e-8fba-fe7c7cd63fb8 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:07.077675 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f6b61-7333-43dc-957f-17f62825eeb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:07.078645 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f6b61-7333-43dc-957f-17f62825eeb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:07.079063 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f6b61-7333-43dc-957f-17f62825eeb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:07.079424 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f6b61-7333-43dc-957f-17f62825eeb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:07.090919 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f6b61-7333-43dc-957f-17f62825eeb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:07.092622 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f6b61-7333-43dc-957f-17f62825eeb1 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:07.109329 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1221/2469] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:34:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:34:07.623660 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a00b007-3b05-4e85-a1ea-5836eac21a2d None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:07.625349 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-6a00b007-3b05-4e85-a1ea-5836eac21a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:07.644900 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a00b007-3b05-4e85-a1ea-5836eac21a2d None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:07.653434 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c729b75-98ed-4267-b477-41752b4167bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:07.655506 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c729b75-98ed-4267-b477-41752b4167bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:07.661996 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c729b75-98ed-4267-b477-41752b4167bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:07.662693 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c729b75-98ed-4267-b477-41752b4167bc admin admin] PATH_INFO: `/v3/projects/f87ba55c18874e4e83ced262286d2d32` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:07.672057 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c729b75-98ed-4267-b477-41752b4167bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f87ba55c18874e4e83ced262286d2d32)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:07.672057 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c729b75-98ed-4267-b477-41752b4167bc admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:07.837874 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-2c729b75-98ed-4267-b477-41752b4167bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f87ba55c18874e4e83ced262286d2d32', 'request_id': 'req-2c729b75-98ed-4267-b477-41752b4167bc'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:07.847009 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-2c729b75-98ed-4267-b477-41752b4167bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f87ba55c18874e4e83ced262286d2d32', 'request_id': 'req-2c729b75-98ed-4267-b477-41752b4167bc'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:07.939537 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-2c729b75-98ed-4267-b477-41752b4167bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f87ba55c18874e4e83ced262286d2d32', 'request_id': 'req-2c729b75-98ed-4267-b477-41752b4167bc'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:07.955625 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-2c729b75-98ed-4267-b477-41752b4167bc admin admin] The token cache is being invalidate because project f87ba55c18874e4e83ced262286d2d32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:34:07.956220 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-2c729b75-98ed-4267-b477-41752b4167bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:07.957226 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-2c729b75-98ed-4267-b477-41752b4167bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:07.962180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1249/2470] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:34:07 2022] DELETE /identity/v3/projects/f87ba55c18874e4e83ced262286d2d32 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:34:07.964192 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f6b61-7333-43dc-957f-17f62825eeb1 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:07.987856 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f6b61-7333-43dc-957f-17f62825eeb1 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:08.025017 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:34:08.079174 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99be7cc-b98e-4ff4-b426-514369b1b6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:08.080200 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99be7cc-b98e-4ff4-b426-514369b1b6fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:08.080646 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99be7cc-b98e-4ff4-b426-514369b1b6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:08.080926 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99be7cc-b98e-4ff4-b426-514369b1b6fc admin admin] PATH_INFO: `/v3/users/be9e4b46b1d94f1183ddbdf8f7ac5c50` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:08.082588 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99be7cc-b98e-4ff4-b426-514369b1b6fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=be9e4b46b1d94f1183ddbdf8f7ac5c50)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:08.084926 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99be7cc-b98e-4ff4-b426-514369b1b6fc admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:08.253397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-c99be7cc-b98e-4ff4-b426-514369b1b6fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be9e4b46b1d94f1183ddbdf8f7ac5c50'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:08.263098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-c99be7cc-b98e-4ff4-b426-514369b1b6fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be9e4b46b1d94f1183ddbdf8f7ac5c50'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:08.282093 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-c99be7cc-b98e-4ff4-b426-514369b1b6fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be9e4b46b1d94f1183ddbdf8f7ac5c50'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:08.282974 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-c99be7cc-b98e-4ff4-b426-514369b1b6fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be9e4b46b1d94f1183ddbdf8f7ac5c50'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:08.283652 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-c99be7cc-b98e-4ff4-b426-514369b1b6fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be9e4b46b1d94f1183ddbdf8f7ac5c50'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:08.353969 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1222/2471] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:34:07 2022] DELETE /identity/v3/users/be9e4b46b1d94f1183ddbdf8f7ac5c50 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:34:08.944774 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c729b75-98ed-4267-b477-41752b4167bc admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:08.950523 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-2c729b75-98ed-4267-b477-41752b4167bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:08.970408 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c729b75-98ed-4267-b477-41752b4167bc admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:09.073581 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e5fd44-7fad-4bab-8e43-197d901f6852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:09.075512 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5fd44-7fad-4bab-8e43-197d901f6852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:09.075783 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5fd44-7fad-4bab-8e43-197d901f6852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:09.076094 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5fd44-7fad-4bab-8e43-197d901f6852 admin admin] PATH_INFO: `/v3/projects/3272e915599e47afa1b5cc3f8ea604c4` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:09.080639 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e5fd44-7fad-4bab-8e43-197d901f6852 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3272e915599e47afa1b5cc3f8ea604c4)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:09.082952 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e5fd44-7fad-4bab-8e43-197d901f6852 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:09.212434 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-87e5fd44-7fad-4bab-8e43-197d901f6852 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3272e915599e47afa1b5cc3f8ea604c4', 'request_id': 'req-87e5fd44-7fad-4bab-8e43-197d901f6852'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:09.222812 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-87e5fd44-7fad-4bab-8e43-197d901f6852 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3272e915599e47afa1b5cc3f8ea604c4', 'request_id': 'req-87e5fd44-7fad-4bab-8e43-197d901f6852'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:09.316382 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-87e5fd44-7fad-4bab-8e43-197d901f6852 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3272e915599e47afa1b5cc3f8ea604c4', 'request_id': 'req-87e5fd44-7fad-4bab-8e43-197d901f6852'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:09.320209 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:34:09.333093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-87e5fd44-7fad-4bab-8e43-197d901f6852 admin admin] The token cache is being invalidate because project 3272e915599e47afa1b5cc3f8ea604c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:34:09.333448 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-87e5fd44-7fad-4bab-8e43-197d901f6852 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:09.334306 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-87e5fd44-7fad-4bab-8e43-197d901f6852 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:09.338112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1250/2472] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:34:08 2022] DELETE /identity/v3/projects/3272e915599e47afa1b5cc3f8ea604c4 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:34:09.343998 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf37f30-e2c8-4337-84eb-5628dab0e90c None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:09.344573 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf37f30-e2c8-4337-84eb-5628dab0e90c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:09.344664 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf37f30-e2c8-4337-84eb-5628dab0e90c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:09.364373 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-7cf37f30-e2c8-4337-84eb-5628dab0e90c None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:34:09.365662 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7cf37f30-e2c8-4337-84eb-5628dab0e90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:09.430758 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1223/2473] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:34:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:34:09.439436 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae3813-5f1e-4376-8088-c30bb652f0fd None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:09.439802 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae3813-5f1e-4376-8088-c30bb652f0fd None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:09.439984 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae3813-5f1e-4376-8088-c30bb652f0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:09.461413 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-e4ae3813-5f1e-4376-8088-c30bb652f0fd None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:34:09.463101 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e4ae3813-5f1e-4376-8088-c30bb652f0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:09.473797 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1251/2474] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:34:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:34:09.478822 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf37f30-e2c8-4337-84eb-5628dab0e90c None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:09.490342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf37f30-e2c8-4337-84eb-5628dab0e90c None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:09.495211 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38741b9c-5cd7-4d40-8179-137b7395d511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:09.495892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38741b9c-5cd7-4d40-8179-137b7395d511 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:09.496211 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38741b9c-5cd7-4d40-8179-137b7395d511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:09.496525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38741b9c-5cd7-4d40-8179-137b7395d511 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:09.497363 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38741b9c-5cd7-4d40-8179-137b7395d511 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:34:09.497681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38741b9c-5cd7-4d40-8179-137b7395d511 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:09.499279 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38741b9c-5cd7-4d40-8179-137b7395d511 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:09.513677 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1224/2475] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:34:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:34:09.518283 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae3813-5f1e-4376-8088-c30bb652f0fd None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:09.529243 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae3813-5f1e-4376-8088-c30bb652f0fd None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:09.534726 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef283c02-29d8-4585-ac34-1de8939d1a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:09.535541 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef283c02-29d8-4585-ac34-1de8939d1a41 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:09.535850 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef283c02-29d8-4585-ac34-1de8939d1a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:09.536171 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef283c02-29d8-4585-ac34-1de8939d1a41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:09.537051 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef283c02-29d8-4585-ac34-1de8939d1a41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:09.539107 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef283c02-29d8-4585-ac34-1de8939d1a41 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:09.678621 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1252/2476] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:34:09 2022] POST /identity/v3/projects => generated 399 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:34:09.680145 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38741b9c-5cd7-4d40-8179-137b7395d511 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:09.690937 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38741b9c-5cd7-4d40-8179-137b7395d511 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:09.724334 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:34:09.758324 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb19e7e4-bf5b-4d92-8b83-167d46863f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:09.759016 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19e7e4-bf5b-4d92-8b83-167d46863f00 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:09.759310 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19e7e4-bf5b-4d92-8b83-167d46863f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:09.759596 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19e7e4-bf5b-4d92-8b83-167d46863f00 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:09.760598 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb19e7e4-bf5b-4d92-8b83-167d46863f00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:09.762516 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb19e7e4-bf5b-4d92-8b83-167d46863f00 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:09.888236 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1225/2477] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:34:09 2022] POST /identity/v3/users => generated 300 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:34:09.891098 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef283c02-29d8-4585-ac34-1de8939d1a41 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:09.921457 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef283c02-29d8-4585-ac34-1de8939d1a41 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:09.928348 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7fba90-0ad1-46db-947d-43a62c800c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:09.929375 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7fba90-0ad1-46db-947d-43a62c800c03 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:09.929694 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7fba90-0ad1-46db-947d-43a62c800c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:09.929976 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7fba90-0ad1-46db-947d-43a62c800c03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:09.930679 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7fba90-0ad1-46db-947d-43a62c800c03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:09.932373 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7fba90-0ad1-46db-947d-43a62c800c03 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:09.957394 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1253/2478] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:34:09.962182 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb19e7e4-bf5b-4d92-8b83-167d46863f00 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:09.979713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb19e7e4-bf5b-4d92-8b83-167d46863f00 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:09.985635 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f09a7d-bb36-49e8-9337-0c0855d41f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:09.986337 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f09a7d-bb36-49e8-9337-0c0855d41f89 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:09.986638 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f09a7d-bb36-49e8-9337-0c0855d41f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:09.986928 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f09a7d-bb36-49e8-9337-0c0855d41f89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:09.987700 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f09a7d-bb36-49e8-9337-0c0855d41f89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:09.989602 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f09a7d-bb36-49e8-9337-0c0855d41f89 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:10.003554 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1226/2479] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 06:34:10.009789 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7fba90-0ad1-46db-947d-43a62c800c03 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:10.089593 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7fba90-0ad1-46db-947d-43a62c800c03 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:10.089593 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b586784-7e62-44f0-a567-403c30de5f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:10.089593 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b586784-7e62-44f0-a567-403c30de5f51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:10.089593 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b586784-7e62-44f0-a567-403c30de5f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:10.091173 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b586784-7e62-44f0-a567-403c30de5f51 admin admin] PATH_INFO: `/v3/projects/688f4e77f7a94e5db7158defe4e77fe6/users/ad0b5bb494224c92bb77b545c266742b/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:10.115012 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b586784-7e62-44f0-a567-403c30de5f51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=688f4e77f7a94e5db7158defe4e77fe6, user_id=ad0b5bb494224c92bb77b545c266742b, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:10.118637 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b586784-7e62-44f0-a567-403c30de5f51 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:10.364344 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1254/2480] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:34:10 2022] PUT /identity/v3/projects/688f4e77f7a94e5db7158defe4e77fe6/users/ad0b5bb494224c92bb77b545c266742b/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:34:10.376322 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6011b9-2a40-4794-a184-70caccb7003d None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:10.377265 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6011b9-2a40-4794-a184-70caccb7003d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:10.377711 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6011b9-2a40-4794-a184-70caccb7003d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:10.417931 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-0e6011b9-2a40-4794-a184-70caccb7003d None None] MFA Rules not processed for user `ad0b5bb494224c92bb77b545c266742b`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:34:10.419536 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-0e6011b9-2a40-4794-a184-70caccb7003d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:10.550640 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1227/2481] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:34:10 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:34:10.555730 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b586784-7e62-44f0-a567-403c30de5f51 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:10.669738 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b586784-7e62-44f0-a567-403c30de5f51 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:10.690201 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c7ad88-39c6-48bb-8363-26a60a657741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:10.690201 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7ad88-39c6-48bb-8363-26a60a657741 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:10.690201 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7ad88-39c6-48bb-8363-26a60a657741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:10.690201 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7ad88-39c6-48bb-8363-26a60a657741 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:10.691010 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c7ad88-39c6-48bb-8363-26a60a657741 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:10.691010 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c7ad88-39c6-48bb-8363-26a60a657741 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:10.808022 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1255/2482] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:34:10 2022] POST /identity/v3/projects => generated 399 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:34:10.813057 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6011b9-2a40-4794-a184-70caccb7003d None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:10.825620 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6011b9-2a40-4794-a184-70caccb7003d None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:10.899003 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0483b6d3-2a1b-41c7-a194-0baced24f2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:10.899703 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483b6d3-2a1b-41c7-a194-0baced24f2e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:10.901290 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483b6d3-2a1b-41c7-a194-0baced24f2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:10.901290 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483b6d3-2a1b-41c7-a194-0baced24f2e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:10.901290 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0483b6d3-2a1b-41c7-a194-0baced24f2e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:10.903521 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0483b6d3-2a1b-41c7-a194-0baced24f2e5 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:11.002433 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1228/2483] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:34:10 2022] POST /identity/v3/users => generated 306 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:34:11.006382 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c7ad88-39c6-48bb-8363-26a60a657741 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:11.017694 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c7ad88-39c6-48bb-8363-26a60a657741 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:11.022735 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f4e88-45e9-4055-bb99-94d40c0c6ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:11.023427 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f4e88-45e9-4055-bb99-94d40c0c6ff5 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:11.023713 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f4e88-45e9-4055-bb99-94d40c0c6ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:11.023999 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f4e88-45e9-4055-bb99-94d40c0c6ff5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:11.024787 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33f4e88-45e9-4055-bb99-94d40c0c6ff5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:11.026452 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33f4e88-45e9-4055-bb99-94d40c0c6ff5 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:11.040407 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1256/2484] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:34:11.046590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0483b6d3-2a1b-41c7-a194-0baced24f2e5 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:11.059493 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0483b6d3-2a1b-41c7-a194-0baced24f2e5 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:11.064347 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1bf70a-7fa3-4695-820d-92c250c946d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:11.065280 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1bf70a-7fa3-4695-820d-92c250c946d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:11.065577 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1bf70a-7fa3-4695-820d-92c250c946d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:11.065859 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1bf70a-7fa3-4695-820d-92c250c946d3 admin admin] PATH_INFO: `/v3/projects/c9e69726062b4177b2c529fa08b602eb/users/51a26366253a4b6d8eb4d62a0679eb1d/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:11.086978 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1bf70a-7fa3-4695-820d-92c250c946d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9e69726062b4177b2c529fa08b602eb, user_id=51a26366253a4b6d8eb4d62a0679eb1d, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:11.092650 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1bf70a-7fa3-4695-820d-92c250c946d3 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:11.179790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1229/2485] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:34:11 2022] PUT /identity/v3/projects/c9e69726062b4177b2c529fa08b602eb/users/51a26366253a4b6d8eb4d62a0679eb1d/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:34:11.185002 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f4e88-45e9-4055-bb99-94d40c0c6ff5 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:11.194333 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f4e88-45e9-4055-bb99-94d40c0c6ff5 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:11.262691 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe2b9d-ec7e-4407-a6b3-ea8ed692abdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:11.263108 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe2b9d-ec7e-4407-a6b3-ea8ed692abdc admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:11.263402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe2b9d-ec7e-4407-a6b3-ea8ed692abdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:11.263689 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe2b9d-ec7e-4407-a6b3-ea8ed692abdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:11.264473 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fe2b9d-ec7e-4407-a6b3-ea8ed692abdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:11.266153 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fe2b9d-ec7e-4407-a6b3-ea8ed692abdc admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:11.269454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:34:11.279132 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1257/2486] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:34:11.286675 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1bf70a-7fa3-4695-820d-92c250c946d3 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:11.292668 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:34:11.299127 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1bf70a-7fa3-4695-820d-92c250c946d3 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:11.304207 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97677758-2852-4da2-9f14-f77fb8340eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:11.305161 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97677758-2852-4da2-9f14-f77fb8340eba admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:11.305454 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97677758-2852-4da2-9f14-f77fb8340eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:11.305743 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97677758-2852-4da2-9f14-f77fb8340eba admin admin] PATH_INFO: `/v3/projects/c9e69726062b4177b2c529fa08b602eb/users/51a26366253a4b6d8eb4d62a0679eb1d/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:11.308082 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97677758-2852-4da2-9f14-f77fb8340eba admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9e69726062b4177b2c529fa08b602eb, user_id=51a26366253a4b6d8eb4d62a0679eb1d, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:11.310792 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97677758-2852-4da2-9f14-f77fb8340eba admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:11.321937 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1230/2487] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:34:11 2022] PUT /identity/v3/projects/c9e69726062b4177b2c529fa08b602eb/users/51a26366253a4b6d8eb4d62a0679eb1d/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:34:11.334611 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6b5c8-9f41-47d6-83a1-56823d5da819 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:11.334912 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6b5c8-9f41-47d6-83a1-56823d5da819 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:11.335199 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6b5c8-9f41-47d6-83a1-56823d5da819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:11.368220 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-24c6b5c8-9f41-47d6-83a1-56823d5da819 None None] MFA Rules not processed for user `51a26366253a4b6d8eb4d62a0679eb1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:34:11.369500 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-24c6b5c8-9f41-47d6-83a1-56823d5da819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:11.476992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1258/2488] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:34:11 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:34:11.489570 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97677758-2852-4da2-9f14-f77fb8340eba admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:11.491489 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-97677758-2852-4da2-9f14-f77fb8340eba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:11.569903 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97677758-2852-4da2-9f14-f77fb8340eba admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:11.577113 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f94f3-82ba-4a51-bb36-5eff70515e77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:11.578122 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f94f3-82ba-4a51-bb36-5eff70515e77 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:11.578515 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f94f3-82ba-4a51-bb36-5eff70515e77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:11.578972 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f94f3-82ba-4a51-bb36-5eff70515e77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:11.589790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673f94f3-82ba-4a51-bb36-5eff70515e77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:11.591735 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673f94f3-82ba-4a51-bb36-5eff70515e77 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:11.604562 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1231/2489] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:34:11 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 06:34:11.845717 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6b5c8-9f41-47d6-83a1-56823d5da819 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:11.860341 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6b5c8-9f41-47d6-83a1-56823d5da819 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:11.864109 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c44b3-a1d1-4fc7-9e65-aa098b51a18b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:11.865401 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c44b3-a1d1-4fc7-9e65-aa098b51a18b service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:11.865401 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c44b3-a1d1-4fc7-9e65-aa098b51a18b service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:11.865401 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c44b3-a1d1-4fc7-9e65-aa098b51a18b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:11.953002 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109c44b3-a1d1-4fc7-9e65-aa098b51a18b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:11.954530 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109c44b3-a1d1-4fc7-9e65-aa098b51a18b service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:11.968564 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1259/2490] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:34:11 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:34:16.516340 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11555d6d-fe87-415b-b203-c48c7da1a877 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:16.517007 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11555d6d-fe87-415b-b203-c48c7da1a877 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:16.517007 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11555d6d-fe87-415b-b203-c48c7da1a877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:16.535406 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-11555d6d-fe87-415b-b203-c48c7da1a877 None None] MFA Rules not processed for user `936ea0959e924aac95cf474687e35c78`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:34:16.536662 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-11555d6d-fe87-415b-b203-c48c7da1a877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:16.604980 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1232/2491] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:34:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:34:18.647492 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c44b3-a1d1-4fc7-9e65-aa098b51a18b service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:18.741879 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c44b3-a1d1-4fc7-9e65-aa098b51a18b service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:18.746816 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a0095-add7-4013-946f-2af73e7484eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:18.747756 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a0095-add7-4013-946f-2af73e7484eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:18.748038 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a0095-add7-4013-946f-2af73e7484eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:18.748311 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a0095-add7-4013-946f-2af73e7484eb admin admin] PATH_INFO: `/v3/users/ad0b5bb494224c92bb77b545c266742b` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:18.749855 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a0095-add7-4013-946f-2af73e7484eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad0b5bb494224c92bb77b545c266742b)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:18.751929 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a0095-add7-4013-946f-2af73e7484eb admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:18.783752 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:34:18.924346 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6a5a0095-add7-4013-946f-2af73e7484eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad0b5bb494224c92bb77b545c266742b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:18.931215 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6a5a0095-add7-4013-946f-2af73e7484eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad0b5bb494224c92bb77b545c266742b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:18.931898 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6a5a0095-add7-4013-946f-2af73e7484eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad0b5bb494224c92bb77b545c266742b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:18.932576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6a5a0095-add7-4013-946f-2af73e7484eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad0b5bb494224c92bb77b545c266742b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:18.945304 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6a5a0095-add7-4013-946f-2af73e7484eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad0b5bb494224c92bb77b545c266742b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:19.043750 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1260/2492] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:34:18 2022] DELETE /identity/v3/users/ad0b5bb494224c92bb77b545c266742b => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:34:19.049056 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf33ecb-a22c-4a80-9a47-4d4346c2c3b8 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:19.049592 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf33ecb-a22c-4a80-9a47-4d4346c2c3b8 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:19.049592 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf33ecb-a22c-4a80-9a47-4d4346c2c3b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:19.069913 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-baf33ecb-a22c-4a80-9a47-4d4346c2c3b8 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:34:19.071596 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-baf33ecb-a22c-4a80-9a47-4d4346c2c3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:19.138985 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1233/2493] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:34:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:34:19.149649 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a0095-add7-4013-946f-2af73e7484eb admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:19.151088 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-6a5a0095-add7-4013-946f-2af73e7484eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:19.237926 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a0095-add7-4013-946f-2af73e7484eb admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:19.241867 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32764e13-c291-4a6d-937c-b779f86e76b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:19.242825 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32764e13-c291-4a6d-937c-b779f86e76b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:19.243126 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32764e13-c291-4a6d-937c-b779f86e76b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:19.243408 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32764e13-c291-4a6d-937c-b779f86e76b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:19.253088 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32764e13-c291-4a6d-937c-b779f86e76b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:19.254570 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32764e13-c291-4a6d-937c-b779f86e76b8 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:19.265348 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1261/2494] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:34:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:34:19.791270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6c4dd-f549-457e-9c70-1b5a90624d36 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:19.791648 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6c4dd-f549-457e-9c70-1b5a90624d36 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:19.792031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6c4dd-f549-457e-9c70-1b5a90624d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:19.819445 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-6ae6c4dd-f549-457e-9c70-1b5a90624d36 None None] MFA Rules not processed for user `683c8a9abae1455bb2ec773b7608d03c`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:34:19.820712 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-6ae6c4dd-f549-457e-9c70-1b5a90624d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:19.862971 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:34:19.889151 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1234/2495] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:34:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:34:19.937257 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32764e13-c291-4a6d-937c-b779f86e76b8 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:19.939132 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-32764e13-c291-4a6d-937c-b779f86e76b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:19.955425 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32764e13-c291-4a6d-937c-b779f86e76b8 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:19.965066 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309e29c2-0b66-4766-94f6-004f0eceb11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:19.968708 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e29c2-0b66-4766-94f6-004f0eceb11c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:19.969001 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e29c2-0b66-4766-94f6-004f0eceb11c admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:19.969146 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e29c2-0b66-4766-94f6-004f0eceb11c admin admin] PATH_INFO: `/v3/projects/688f4e77f7a94e5db7158defe4e77fe6` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:19.986205 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309e29c2-0b66-4766-94f6-004f0eceb11c admin admin] RBAC: Authorizing `identity:delete_project(project_id=688f4e77f7a94e5db7158defe4e77fe6)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:19.991426 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309e29c2-0b66-4766-94f6-004f0eceb11c admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:20.178602 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-309e29c2-0b66-4766-94f6-004f0eceb11c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '688f4e77f7a94e5db7158defe4e77fe6', 'request_id': 'req-309e29c2-0b66-4766-94f6-004f0eceb11c'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:20.189750 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-309e29c2-0b66-4766-94f6-004f0eceb11c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '688f4e77f7a94e5db7158defe4e77fe6', 'request_id': 'req-309e29c2-0b66-4766-94f6-004f0eceb11c'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:20.278280 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-309e29c2-0b66-4766-94f6-004f0eceb11c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '688f4e77f7a94e5db7158defe4e77fe6', 'request_id': 'req-309e29c2-0b66-4766-94f6-004f0eceb11c'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:20.292334 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-309e29c2-0b66-4766-94f6-004f0eceb11c admin admin] The token cache is being invalidate because project 688f4e77f7a94e5db7158defe4e77fe6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:34:20.292334 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-309e29c2-0b66-4766-94f6-004f0eceb11c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:20.293084 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-309e29c2-0b66-4766-94f6-004f0eceb11c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:20.295094 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1262/2496] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:34:19 2022] DELETE /identity/v3/projects/688f4e77f7a94e5db7158defe4e77fe6 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:34:20.299476 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae6c4dd-f549-457e-9c70-1b5a90624d36 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:20.313442 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae6c4dd-f549-457e-9c70-1b5a90624d36 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:20.381016 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f9ec1-4b9d-470a-8261-9be87ec4d2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:20.382283 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f9ec1-4b9d-470a-8261-9be87ec4d2b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:20.382590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f9ec1-4b9d-470a-8261-9be87ec4d2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:20.383907 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f9ec1-4b9d-470a-8261-9be87ec4d2b7 admin admin] PATH_INFO: `/v3/users/51a26366253a4b6d8eb4d62a0679eb1d` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:20.385412 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f9ec1-4b9d-470a-8261-9be87ec4d2b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51a26366253a4b6d8eb4d62a0679eb1d)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:20.387882 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f9ec1-4b9d-470a-8261-9be87ec4d2b7 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:20.605366 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0c6f9ec1-4b9d-470a-8261-9be87ec4d2b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51a26366253a4b6d8eb4d62a0679eb1d'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:20.613463 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0c6f9ec1-4b9d-470a-8261-9be87ec4d2b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51a26366253a4b6d8eb4d62a0679eb1d'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:20.627005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0c6f9ec1-4b9d-470a-8261-9be87ec4d2b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51a26366253a4b6d8eb4d62a0679eb1d'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:20.637354 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0c6f9ec1-4b9d-470a-8261-9be87ec4d2b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51a26366253a4b6d8eb4d62a0679eb1d'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:20.637354 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0c6f9ec1-4b9d-470a-8261-9be87ec4d2b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51a26366253a4b6d8eb4d62a0679eb1d'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:20.714026 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1235/2497] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:34:20 2022] DELETE /identity/v3/users/51a26366253a4b6d8eb4d62a0679eb1d => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:34:21.261408 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309e29c2-0b66-4766-94f6-004f0eceb11c admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:21.263105 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-309e29c2-0b66-4766-94f6-004f0eceb11c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:21.291220 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309e29c2-0b66-4766-94f6-004f0eceb11c admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:21.317215 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:34:21.429280 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d845e-84c8-4aba-b885-613063077a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:21.430998 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d845e-84c8-4aba-b885-613063077a43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:21.431341 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d845e-84c8-4aba-b885-613063077a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:21.431859 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d845e-84c8-4aba-b885-613063077a43 admin admin] PATH_INFO: `/v3/projects/c9e69726062b4177b2c529fa08b602eb` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:21.434184 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d845e-84c8-4aba-b885-613063077a43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9e69726062b4177b2c529fa08b602eb)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:21.438694 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d845e-84c8-4aba-b885-613063077a43 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:21.586049 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-703d845e-84c8-4aba-b885-613063077a43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9e69726062b4177b2c529fa08b602eb', 'request_id': 'req-703d845e-84c8-4aba-b885-613063077a43'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:21.594861 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-703d845e-84c8-4aba-b885-613063077a43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9e69726062b4177b2c529fa08b602eb', 'request_id': 'req-703d845e-84c8-4aba-b885-613063077a43'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:21.738492 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-703d845e-84c8-4aba-b885-613063077a43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9e69726062b4177b2c529fa08b602eb', 'request_id': 'req-703d845e-84c8-4aba-b885-613063077a43'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:21.751123 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-703d845e-84c8-4aba-b885-613063077a43 admin admin] The token cache is being invalidate because project c9e69726062b4177b2c529fa08b602eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:34:21.751413 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-703d845e-84c8-4aba-b885-613063077a43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:21.752077 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-703d845e-84c8-4aba-b885-613063077a43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:21.753656 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1263/2498] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:34:21 2022] DELETE /identity/v3/projects/c9e69726062b4177b2c529fa08b602eb => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:34:21.758777 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b56d67-05b6-4c2a-8959-d2fa56ac6d5a None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:21.759350 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b56d67-05b6-4c2a-8959-d2fa56ac6d5a None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:21.759350 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b56d67-05b6-4c2a-8959-d2fa56ac6d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:21.779352 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-55b56d67-05b6-4c2a-8959-d2fa56ac6d5a None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:34:21.781340 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-55b56d67-05b6-4c2a-8959-d2fa56ac6d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:21.847462 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1236/2499] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:34:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:34:21.855912 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7701b-f4ca-4670-9f7e-e7dcef424fda None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:21.856964 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7701b-f4ca-4670-9f7e-e7dcef424fda None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:21.856964 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7701b-f4ca-4670-9f7e-e7dcef424fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:21.888033 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-7ac7701b-f4ca-4670-9f7e-e7dcef424fda None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:34:21.889336 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-7ac7701b-f4ca-4670-9f7e-e7dcef424fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:21.899801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1264/2500] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:34:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:34:21.906159 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b56d67-05b6-4c2a-8959-d2fa56ac6d5a None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:21.912088 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:34:21.919337 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b56d67-05b6-4c2a-8959-d2fa56ac6d5a None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:21.928736 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9701cf-bf1b-4198-b47d-ac67d7688956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:21.930110 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9701cf-bf1b-4198-b47d-ac67d7688956 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:21.930659 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9701cf-bf1b-4198-b47d-ac67d7688956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:21.931090 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9701cf-bf1b-4198-b47d-ac67d7688956 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:21.932177 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9701cf-bf1b-4198-b47d-ac67d7688956 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:34:21.933584 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9701cf-bf1b-4198-b47d-ac67d7688956 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:21.935398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9701cf-bf1b-4198-b47d-ac67d7688956 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:21.953608 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1237/2501] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:34:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:34:21.961527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7701b-f4ca-4670-9f7e-e7dcef424fda None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:21.975236 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7701b-f4ca-4670-9f7e-e7dcef424fda None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:21.981074 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdebd277-8177-4505-92b9-8a18f3e2a3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:21.981935 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdebd277-8177-4505-92b9-8a18f3e2a3ad admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:21.982254 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdebd277-8177-4505-92b9-8a18f3e2a3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:21.982720 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdebd277-8177-4505-92b9-8a18f3e2a3ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:21.983599 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdebd277-8177-4505-92b9-8a18f3e2a3ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:21.985817 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdebd277-8177-4505-92b9-8a18f3e2a3ad admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:22.103505 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1265/2502] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:34:21 2022] POST /identity/v3/projects => generated 381 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:34:22.104685 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9701cf-bf1b-4198-b47d-ac67d7688956 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:22.116581 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9701cf-bf1b-4198-b47d-ac67d7688956 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:22.194970 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f7093-2025-4bdb-b5cb-f3600f72e2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:22.196183 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f7093-2025-4bdb-b5cb-f3600f72e2ef admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:22.196696 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f7093-2025-4bdb-b5cb-f3600f72e2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:22.197118 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f7093-2025-4bdb-b5cb-f3600f72e2ef admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:22.198009 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768f7093-2025-4bdb-b5cb-f3600f72e2ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:22.200150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768f7093-2025-4bdb-b5cb-f3600f72e2ef admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:22.309109 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1238/2503] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:34:22 2022] POST /identity/v3/users => generated 291 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:34:22.314031 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdebd277-8177-4505-92b9-8a18f3e2a3ad admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:22.327774 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdebd277-8177-4505-92b9-8a18f3e2a3ad admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:22.340389 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc93a55b-e889-412c-9cbc-34116035502e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:22.341460 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc93a55b-e889-412c-9cbc-34116035502e admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:22.341750 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc93a55b-e889-412c-9cbc-34116035502e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:22.342029 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc93a55b-e889-412c-9cbc-34116035502e admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:22.342889 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc93a55b-e889-412c-9cbc-34116035502e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:22.344554 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc93a55b-e889-412c-9cbc-34116035502e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:22.376470 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1266/2504] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:34:22 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:34:22.382272 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f7093-2025-4bdb-b5cb-f3600f72e2ef admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:22.395556 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f7093-2025-4bdb-b5cb-f3600f72e2ef admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:22.404251 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6722d237-4ca4-47b7-a6e7-1eb89d6f0e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:22.404251 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6722d237-4ca4-47b7-a6e7-1eb89d6f0e10 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:22.404251 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6722d237-4ca4-47b7-a6e7-1eb89d6f0e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:22.404888 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6722d237-4ca4-47b7-a6e7-1eb89d6f0e10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:22.406707 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6722d237-4ca4-47b7-a6e7-1eb89d6f0e10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:22.409157 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6722d237-4ca4-47b7-a6e7-1eb89d6f0e10 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:22.426789 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1239/2505] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:34:22 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:34:22.431410 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc93a55b-e889-412c-9cbc-34116035502e admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:22.443643 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc93a55b-e889-412c-9cbc-34116035502e admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:22.449297 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b2ad6b-75f1-4a79-b075-1276fd89f352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:22.450311 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b2ad6b-75f1-4a79-b075-1276fd89f352 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:22.450603 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b2ad6b-75f1-4a79-b075-1276fd89f352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:22.450892 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b2ad6b-75f1-4a79-b075-1276fd89f352 admin admin] PATH_INFO: `/v3/projects/eface5658b9f4ec19590d82da4f2efb4/users/7c5186475dd247ada18d88915bc1a720/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:22.469281 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b2ad6b-75f1-4a79-b075-1276fd89f352 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eface5658b9f4ec19590d82da4f2efb4, user_id=7c5186475dd247ada18d88915bc1a720, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:22.472254 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b2ad6b-75f1-4a79-b075-1276fd89f352 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:22.588469 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1267/2506] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:34:22 2022] PUT /identity/v3/projects/eface5658b9f4ec19590d82da4f2efb4/users/7c5186475dd247ada18d88915bc1a720/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:34:22.599117 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3c370-5c40-463f-989c-5510d43085bc None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:22.599475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3c370-5c40-463f-989c-5510d43085bc None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:22.599754 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3c370-5c40-463f-989c-5510d43085bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:22.630205 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-33d3c370-5c40-463f-989c-5510d43085bc None None] MFA Rules not processed for user `7c5186475dd247ada18d88915bc1a720`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:34:22.633082 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-33d3c370-5c40-463f-989c-5510d43085bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:22.715179 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1240/2507] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:34:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:34:22.720949 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b2ad6b-75f1-4a79-b075-1276fd89f352 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:22.761754 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:34:22.796795 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b2ad6b-75f1-4a79-b075-1276fd89f352 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:22.802172 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197274d6-7628-48be-802a-a1bb8cbe5bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:22.802882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197274d6-7628-48be-802a-a1bb8cbe5bbe admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:22.803176 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197274d6-7628-48be-802a-a1bb8cbe5bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:22.803465 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197274d6-7628-48be-802a-a1bb8cbe5bbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:22.804231 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197274d6-7628-48be-802a-a1bb8cbe5bbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:22.806426 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197274d6-7628-48be-802a-a1bb8cbe5bbe admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:22.946896 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1268/2508] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:34:22 2022] POST /identity/v3/projects => generated 383 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:34:22.948113 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3c370-5c40-463f-989c-5510d43085bc None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:22.959539 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3c370-5c40-463f-989c-5510d43085bc None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:23.016966 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:34:23.023521 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b3aa99-510f-400b-aa88-cdd81d4624b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:23.024211 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3aa99-510f-400b-aa88-cdd81d4624b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:23.024539 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3aa99-510f-400b-aa88-cdd81d4624b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:23.024818 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3aa99-510f-400b-aa88-cdd81d4624b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:23.025951 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b3aa99-510f-400b-aa88-cdd81d4624b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:23.028073 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b3aa99-510f-400b-aa88-cdd81d4624b2 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:23.188007 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1241/2509] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:34:22 2022] POST /identity/v3/users => generated 298 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:34:23.192784 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197274d6-7628-48be-802a-a1bb8cbe5bbe admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:23.206128 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197274d6-7628-48be-802a-a1bb8cbe5bbe admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:23.210334 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81341d5-2452-4bdc-bd85-869568730dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:23.211037 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81341d5-2452-4bdc-bd85-869568730dfe admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:23.211376 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81341d5-2452-4bdc-bd85-869568730dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:23.211642 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81341d5-2452-4bdc-bd85-869568730dfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:23.212403 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81341d5-2452-4bdc-bd85-869568730dfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:23.214085 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81341d5-2452-4bdc-bd85-869568730dfe admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:23.229958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1269/2510] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:34:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:34:23.235138 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b3aa99-510f-400b-aa88-cdd81d4624b2 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:23.246310 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b3aa99-510f-400b-aa88-cdd81d4624b2 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:23.251867 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f76548b-c122-41cb-91d9-85e52975babf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:23.252848 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76548b-c122-41cb-91d9-85e52975babf admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:23.253177 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76548b-c122-41cb-91d9-85e52975babf admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:23.253459 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76548b-c122-41cb-91d9-85e52975babf admin admin] PATH_INFO: `/v3/projects/72f1e931993a4e05a6c72f9f2432dbac/users/f1109506ae7e4e309f32c59b730003c0/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:23.272057 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f76548b-c122-41cb-91d9-85e52975babf admin admin] RBAC: Authorizing `identity:create_grant(project_id=72f1e931993a4e05a6c72f9f2432dbac, user_id=f1109506ae7e4e309f32c59b730003c0, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:23.275005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f76548b-c122-41cb-91d9-85e52975babf admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:23.348905 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1242/2511] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:34:23 2022] PUT /identity/v3/projects/72f1e931993a4e05a6c72f9f2432dbac/users/f1109506ae7e4e309f32c59b730003c0/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:34:23.357155 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81341d5-2452-4bdc-bd85-869568730dfe admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:23.372125 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81341d5-2452-4bdc-bd85-869568730dfe admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:23.450257 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3818b800-3849-4ec1-a185-f08cc0cddd32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:23.453795 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3818b800-3849-4ec1-a185-f08cc0cddd32 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:23.453795 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3818b800-3849-4ec1-a185-f08cc0cddd32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:23.453795 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3818b800-3849-4ec1-a185-f08cc0cddd32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:23.453795 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3818b800-3849-4ec1-a185-f08cc0cddd32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:23.455008 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3818b800-3849-4ec1-a185-f08cc0cddd32 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:23.469764 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1270/2512] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:34:23 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:34:23.470997 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f76548b-c122-41cb-91d9-85e52975babf admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:23.482927 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f76548b-c122-41cb-91d9-85e52975babf admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:23.487871 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a7817-17b3-441a-8af4-281e160b717e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:23.488754 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a7817-17b3-441a-8af4-281e160b717e admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:23.489150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a7817-17b3-441a-8af4-281e160b717e admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:23.489324 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a7817-17b3-441a-8af4-281e160b717e admin admin] PATH_INFO: `/v3/projects/72f1e931993a4e05a6c72f9f2432dbac/users/f1109506ae7e4e309f32c59b730003c0/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:23.491367 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30a7817-17b3-441a-8af4-281e160b717e admin admin] RBAC: Authorizing `identity:create_grant(project_id=72f1e931993a4e05a6c72f9f2432dbac, user_id=f1109506ae7e4e309f32c59b730003c0, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:23.494572 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30a7817-17b3-441a-8af4-281e160b717e admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:23.507790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1243/2513] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:34:23 2022] PUT /identity/v3/projects/72f1e931993a4e05a6c72f9f2432dbac/users/f1109506ae7e4e309f32c59b730003c0/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:34:23.518020 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22fd49a-7018-4941-8f91-d51ec21af2aa None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:23.518482 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22fd49a-7018-4941-8f91-d51ec21af2aa None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:23.518767 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22fd49a-7018-4941-8f91-d51ec21af2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:23.557160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-a22fd49a-7018-4941-8f91-d51ec21af2aa None None] MFA Rules not processed for user `f1109506ae7e4e309f32c59b730003c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:34:23.558173 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-a22fd49a-7018-4941-8f91-d51ec21af2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:23.641264 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1271/2514] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:34:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:34:23.654327 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a7817-17b3-441a-8af4-281e160b717e admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:23.656951 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b30a7817-17b3-441a-8af4-281e160b717e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:23.742324 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a7817-17b3-441a-8af4-281e160b717e admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:23.746237 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a559a019-dc74-4af4-935c-013ee244f99c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:23.746885 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a559a019-dc74-4af4-935c-013ee244f99c service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:23.747175 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a559a019-dc74-4af4-935c-013ee244f99c service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:23.747457 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a559a019-dc74-4af4-935c-013ee244f99c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:23.757925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a559a019-dc74-4af4-935c-013ee244f99c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:23.759379 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a559a019-dc74-4af4-935c-013ee244f99c service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:23.771275 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1244/2515] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:34:23 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:34:23.927053 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22fd49a-7018-4941-8f91-d51ec21af2aa None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:23.937757 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22fd49a-7018-4941-8f91-d51ec21af2aa None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:23.941505 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbcfe24-62de-47eb-b953-d5fc7f020adc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:23.942165 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbcfe24-62de-47eb-b953-d5fc7f020adc service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:23.942454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbcfe24-62de-47eb-b953-d5fc7f020adc service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:23.942735 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbcfe24-62de-47eb-b953-d5fc7f020adc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:23.971270 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:34:24.002728 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbcfe24-62de-47eb-b953-d5fc7f020adc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:24.004089 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbcfe24-62de-47eb-b953-d5fc7f020adc service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:24.013683 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1272/2516] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:34:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:34:24.528106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a559a019-dc74-4af4-935c-013ee244f99c service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:24.598799 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a559a019-dc74-4af4-935c-013ee244f99c service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:24.603382 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c6c54-885b-4544-bcb1-18cbae320e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:24.604294 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c6c54-885b-4544-bcb1-18cbae320e29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:24.604632 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c6c54-885b-4544-bcb1-18cbae320e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:24.604916 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c6c54-885b-4544-bcb1-18cbae320e29 admin admin] PATH_INFO: `/v3/users/7c5186475dd247ada18d88915bc1a720` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:24.606154 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c6c54-885b-4544-bcb1-18cbae320e29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c5186475dd247ada18d88915bc1a720)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:24.608742 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c6c54-885b-4544-bcb1-18cbae320e29 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:24.831500 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:34:24.842325 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e88c6c54-885b-4544-bcb1-18cbae320e29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c5186475dd247ada18d88915bc1a720'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:24.850294 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e88c6c54-885b-4544-bcb1-18cbae320e29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c5186475dd247ada18d88915bc1a720'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:24.863637 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e88c6c54-885b-4544-bcb1-18cbae320e29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c5186475dd247ada18d88915bc1a720'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:24.864613 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e88c6c54-885b-4544-bcb1-18cbae320e29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c5186475dd247ada18d88915bc1a720'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:24.868226 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e88c6c54-885b-4544-bcb1-18cbae320e29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c5186475dd247ada18d88915bc1a720'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:24.956648 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1245/2517] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:34:24 2022] DELETE /identity/v3/users/7c5186475dd247ada18d88915bc1a720 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:34:24.961617 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95dbcb-3732-44e4-a5a0-dff5a7aea336 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:24.962130 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95dbcb-3732-44e4-a5a0-dff5a7aea336 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:24.962225 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95dbcb-3732-44e4-a5a0-dff5a7aea336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:24.982228 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-3e95dbcb-3732-44e4-a5a0-dff5a7aea336 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:34:24.983793 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-3e95dbcb-3732-44e4-a5a0-dff5a7aea336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:25.052441 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1273/2518] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:34:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:34:25.062654 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c6c54-885b-4544-bcb1-18cbae320e29 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:25.064127 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e88c6c54-885b-4544-bcb1-18cbae320e29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:25.125710 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c6c54-885b-4544-bcb1-18cbae320e29 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:25.129333 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b6fdb-7581-4684-87af-e4f3fcfd9704 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:25.129988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b6fdb-7581-4684-87af-e4f3fcfd9704 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:25.130275 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b6fdb-7581-4684-87af-e4f3fcfd9704 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:25.130556 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b6fdb-7581-4684-87af-e4f3fcfd9704 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:25.140920 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39b6fdb-7581-4684-87af-e4f3fcfd9704 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:25.142557 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39b6fdb-7581-4684-87af-e4f3fcfd9704 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:25.153189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1246/2519] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:34:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:34:25.615640 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95dbcb-3732-44e4-a5a0-dff5a7aea336 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:25.617185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-3e95dbcb-3732-44e4-a5a0-dff5a7aea336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:25.630601 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95dbcb-3732-44e4-a5a0-dff5a7aea336 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:25.638147 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015aab99-486d-488f-98da-702a63915630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:25.639186 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015aab99-486d-488f-98da-702a63915630 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:25.640332 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015aab99-486d-488f-98da-702a63915630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:25.640807 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015aab99-486d-488f-98da-702a63915630 admin admin] PATH_INFO: `/v3/projects/eface5658b9f4ec19590d82da4f2efb4` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:25.642448 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015aab99-486d-488f-98da-702a63915630 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eface5658b9f4ec19590d82da4f2efb4)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:25.644640 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015aab99-486d-488f-98da-702a63915630 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:25.799899 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-015aab99-486d-488f-98da-702a63915630 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eface5658b9f4ec19590d82da4f2efb4', 'request_id': 'req-015aab99-486d-488f-98da-702a63915630'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:25.807660 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-015aab99-486d-488f-98da-702a63915630 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eface5658b9f4ec19590d82da4f2efb4', 'request_id': 'req-015aab99-486d-488f-98da-702a63915630'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:25.910448 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-015aab99-486d-488f-98da-702a63915630 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eface5658b9f4ec19590d82da4f2efb4', 'request_id': 'req-015aab99-486d-488f-98da-702a63915630'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:25.926995 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-015aab99-486d-488f-98da-702a63915630 admin admin] The token cache is being invalidate because project eface5658b9f4ec19590d82da4f2efb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:34:25.927329 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-015aab99-486d-488f-98da-702a63915630 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:25.928006 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-015aab99-486d-488f-98da-702a63915630 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:25.929661 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1274/2520] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:34:25 2022] DELETE /identity/v3/projects/eface5658b9f4ec19590d82da4f2efb4 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:34:25.933754 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b6fdb-7581-4684-87af-e4f3fcfd9704 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:25.944160 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b6fdb-7581-4684-87af-e4f3fcfd9704 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:26.010886 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2307e99a-b1bc-4dcb-aad4-d8caf6863820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:26.011807 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307e99a-b1bc-4dcb-aad4-d8caf6863820 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:26.012096 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307e99a-b1bc-4dcb-aad4-d8caf6863820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:26.012377 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307e99a-b1bc-4dcb-aad4-d8caf6863820 admin admin] PATH_INFO: `/v3/users/f1109506ae7e4e309f32c59b730003c0` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:26.013809 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2307e99a-b1bc-4dcb-aad4-d8caf6863820 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1109506ae7e4e309f32c59b730003c0)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:26.016111 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2307e99a-b1bc-4dcb-aad4-d8caf6863820 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:26.293263 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-2307e99a-b1bc-4dcb-aad4-d8caf6863820 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1109506ae7e4e309f32c59b730003c0'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:26.305723 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-2307e99a-b1bc-4dcb-aad4-d8caf6863820 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1109506ae7e4e309f32c59b730003c0'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:26.305723 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:34:26.320589 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-2307e99a-b1bc-4dcb-aad4-d8caf6863820 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1109506ae7e4e309f32c59b730003c0'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:26.321364 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-2307e99a-b1bc-4dcb-aad4-d8caf6863820 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1109506ae7e4e309f32c59b730003c0'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:26.322020 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-2307e99a-b1bc-4dcb-aad4-d8caf6863820 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1109506ae7e4e309f32c59b730003c0'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:26.417315 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1247/2521] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:34:25 2022] DELETE /identity/v3/users/f1109506ae7e4e309f32c59b730003c0 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 06:34:27.183591 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015aab99-486d-488f-98da-702a63915630 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:27.188650 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-015aab99-486d-488f-98da-702a63915630 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:27.208055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015aab99-486d-488f-98da-702a63915630 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:27.307287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:34:27.320038 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb6096-d71f-48d8-9383-c628b8628a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:27.321000 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb6096-d71f-48d8-9383-c628b8628a2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:27.321292 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb6096-d71f-48d8-9383-c628b8628a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:27.321587 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb6096-d71f-48d8-9383-c628b8628a2e admin admin] PATH_INFO: `/v3/projects/72f1e931993a4e05a6c72f9f2432dbac` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:27.323111 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcb6096-d71f-48d8-9383-c628b8628a2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=72f1e931993a4e05a6c72f9f2432dbac)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:27.325223 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcb6096-d71f-48d8-9383-c628b8628a2e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:27.477594 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-9fcb6096-d71f-48d8-9383-c628b8628a2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72f1e931993a4e05a6c72f9f2432dbac', 'request_id': 'req-9fcb6096-d71f-48d8-9383-c628b8628a2e'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:27.490303 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-9fcb6096-d71f-48d8-9383-c628b8628a2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72f1e931993a4e05a6c72f9f2432dbac', 'request_id': 'req-9fcb6096-d71f-48d8-9383-c628b8628a2e'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:27.587206 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-9fcb6096-d71f-48d8-9383-c628b8628a2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72f1e931993a4e05a6c72f9f2432dbac', 'request_id': 'req-9fcb6096-d71f-48d8-9383-c628b8628a2e'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:27.611214 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-9fcb6096-d71f-48d8-9383-c628b8628a2e admin admin] The token cache is being invalidate because project 72f1e931993a4e05a6c72f9f2432dbac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:34:27.612301 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-9fcb6096-d71f-48d8-9383-c628b8628a2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:27.612706 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-9fcb6096-d71f-48d8-9383-c628b8628a2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:27.615450 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1275/2522] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:34:27 2022] DELETE /identity/v3/projects/72f1e931993a4e05a6c72f9f2432dbac => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:34:27.622214 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7042c98-3230-4f00-800c-32051dbbd10b None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:27.628928 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7042c98-3230-4f00-800c-32051dbbd10b None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:27.628928 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7042c98-3230-4f00-800c-32051dbbd10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:27.659906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-c7042c98-3230-4f00-800c-32051dbbd10b None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:34:27.663407 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-c7042c98-3230-4f00-800c-32051dbbd10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:27.740918 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1248/2523] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:34:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:34:27.749216 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d524e8-cf87-485d-8daa-601cbb7b8c18 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:27.749572 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d524e8-cf87-485d-8daa-601cbb7b8c18 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:27.749977 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d524e8-cf87-485d-8daa-601cbb7b8c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:27.771075 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-00d524e8-cf87-485d-8daa-601cbb7b8c18 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:34:27.772609 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-00d524e8-cf87-485d-8daa-601cbb7b8c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:27.781936 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1276/2524] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:34:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:34:27.786629 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7042c98-3230-4f00-800c-32051dbbd10b None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:27.797256 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7042c98-3230-4f00-800c-32051dbbd10b None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:27.801955 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b1677-d16e-4739-a73e-fd49ef532a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:27.802626 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b1677-d16e-4739-a73e-fd49ef532a86 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:27.802915 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b1677-d16e-4739-a73e-fd49ef532a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:27.803232 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b1677-d16e-4739-a73e-fd49ef532a86 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:27.803968 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b1677-d16e-4739-a73e-fd49ef532a86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:34:27.804337 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b1677-d16e-4739-a73e-fd49ef532a86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:27.806003 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b1677-d16e-4739-a73e-fd49ef532a86 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:27.819355 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1249/2525] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:34:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:34:27.824598 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d524e8-cf87-485d-8daa-601cbb7b8c18 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:27.835901 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d524e8-cf87-485d-8daa-601cbb7b8c18 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:27.842088 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d7aab8-69a1-4e9d-9194-5afdde574d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:27.843577 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d7aab8-69a1-4e9d-9194-5afdde574d46 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:27.843902 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d7aab8-69a1-4e9d-9194-5afdde574d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:27.844614 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d7aab8-69a1-4e9d-9194-5afdde574d46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:27.844988 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d7aab8-69a1-4e9d-9194-5afdde574d46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:27.847056 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d7aab8-69a1-4e9d-9194-5afdde574d46 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:28.021443 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1277/2526] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:34:27 2022] POST /identity/v3/projects => generated 375 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:34:28.023909 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b1677-d16e-4739-a73e-fd49ef532a86 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:28.034369 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b1677-d16e-4739-a73e-fd49ef532a86 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:28.143542 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d46ff-fa20-41eb-941b-17723ad895a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:28.144718 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d46ff-fa20-41eb-941b-17723ad895a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:28.145233 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d46ff-fa20-41eb-941b-17723ad895a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:28.145715 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d46ff-fa20-41eb-941b-17723ad895a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:28.147018 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d46ff-fa20-41eb-941b-17723ad895a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:28.151892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d46ff-fa20-41eb-941b-17723ad895a8 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:28.262065 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1250/2527] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:34:28 2022] POST /identity/v3/users => generated 288 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:34:28.264390 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d7aab8-69a1-4e9d-9194-5afdde574d46 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:28.284028 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d7aab8-69a1-4e9d-9194-5afdde574d46 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:28.292222 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e467009d-e96b-42de-babb-4b7a0f06cf78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:28.294076 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467009d-e96b-42de-babb-4b7a0f06cf78 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:28.294438 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467009d-e96b-42de-babb-4b7a0f06cf78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:28.294759 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467009d-e96b-42de-babb-4b7a0f06cf78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:28.296994 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e467009d-e96b-42de-babb-4b7a0f06cf78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:28.299502 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e467009d-e96b-42de-babb-4b7a0f06cf78 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:28.313511 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1278/2528] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:34:28 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:34:28.317832 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d46ff-fa20-41eb-941b-17723ad895a8 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:28.329686 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d46ff-fa20-41eb-941b-17723ad895a8 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:28.335287 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554cd4ce-797a-42d7-b8f0-99ef50db9f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:28.335977 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554cd4ce-797a-42d7-b8f0-99ef50db9f8b admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:28.336263 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554cd4ce-797a-42d7-b8f0-99ef50db9f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:28.336575 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554cd4ce-797a-42d7-b8f0-99ef50db9f8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:28.337353 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554cd4ce-797a-42d7-b8f0-99ef50db9f8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:28.338975 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554cd4ce-797a-42d7-b8f0-99ef50db9f8b admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:28.351248 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1251/2529] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:34:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:34:28.355329 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e467009d-e96b-42de-babb-4b7a0f06cf78 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:28.368793 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e467009d-e96b-42de-babb-4b7a0f06cf78 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:28.373626 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065b926-5063-4975-8a73-c338572290bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:28.374619 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065b926-5063-4975-8a73-c338572290bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:28.374913 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065b926-5063-4975-8a73-c338572290bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:28.375197 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065b926-5063-4975-8a73-c338572290bd admin admin] PATH_INFO: `/v3/projects/63378d84e46c4737b89aa861626f155c/users/5c5c74e875c24bf694e89862f856dbb3/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:28.392070 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1065b926-5063-4975-8a73-c338572290bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=63378d84e46c4737b89aa861626f155c, user_id=5c5c74e875c24bf694e89862f856dbb3, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:28.394934 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1065b926-5063-4975-8a73-c338572290bd admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:28.498243 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1279/2530] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:34:28 2022] PUT /identity/v3/projects/63378d84e46c4737b89aa861626f155c/users/5c5c74e875c24bf694e89862f856dbb3/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:34:28.509645 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f13da-9da9-4514-bd9b-7bfa2ab4e7ec None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:28.509944 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f13da-9da9-4514-bd9b-7bfa2ab4e7ec None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:28.510279 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f13da-9da9-4514-bd9b-7bfa2ab4e7ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:28.547219 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-b73f13da-9da9-4514-bd9b-7bfa2ab4e7ec None None] MFA Rules not processed for user `5c5c74e875c24bf694e89862f856dbb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:34:28.548508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b73f13da-9da9-4514-bd9b-7bfa2ab4e7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:28.586968 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:34:28.643371 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1252/2531] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:34:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:34:28.654773 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065b926-5063-4975-8a73-c338572290bd admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:28.656910 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-1065b926-5063-4975-8a73-c338572290bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:28.746648 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065b926-5063-4975-8a73-c338572290bd admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:28.754126 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f9b48e-f489-4d1c-94d4-fb2314cca5bb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1193330cb5644765a7abb77738929ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:28.754954 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9b48e-f489-4d1c-94d4-fb2314cca5bb service glance] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:28.755339 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9b48e-f489-4d1c-94d4-fb2314cca5bb service glance] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:28.755844 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9b48e-f489-4d1c-94d4-fb2314cca5bb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:28.776761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f9b48e-f489-4d1c-94d4-fb2314cca5bb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:28.778777 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f9b48e-f489-4d1c-94d4-fb2314cca5bb service glance] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:28.789545 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1280/2532] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:34:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:34:38.784756 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=81,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:34:38.853218 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=14 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:34:57.612899 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f13da-9da9-4514-bd9b-7bfa2ab4e7ec None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:57.678985 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f13da-9da9-4514-bd9b-7bfa2ab4e7ec None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:57.683666 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a91cc1-23cb-439e-bc80-d19957920e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:57.684624 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a91cc1-23cb-439e-bc80-d19957920e09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:57.684914 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a91cc1-23cb-439e-bc80-d19957920e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:57.685226 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a91cc1-23cb-439e-bc80-d19957920e09 admin admin] PATH_INFO: `/v3/users/5c5c74e875c24bf694e89862f856dbb3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:57.686471 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a91cc1-23cb-439e-bc80-d19957920e09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c5c74e875c24bf694e89862f856dbb3)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:57.688398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a91cc1-23cb-439e-bc80-d19957920e09 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:57.878243 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-40a91cc1-23cb-439e-bc80-d19957920e09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c5c74e875c24bf694e89862f856dbb3'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:57.885366 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-40a91cc1-23cb-439e-bc80-d19957920e09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c5c74e875c24bf694e89862f856dbb3'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:57.897349 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-40a91cc1-23cb-439e-bc80-d19957920e09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c5c74e875c24bf694e89862f856dbb3'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:57.898077 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-40a91cc1-23cb-439e-bc80-d19957920e09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c5c74e875c24bf694e89862f856dbb3'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:57.898690 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-40a91cc1-23cb-439e-bc80-d19957920e09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c5c74e875c24bf694e89862f856dbb3'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:57.980598 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1253/2533] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:34:57 2022] DELETE /identity/v3/users/5c5c74e875c24bf694e89862f856dbb3 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:34:57.985147 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09dc24-91f2-46f0-bb44-95576fc77236 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:57.994319 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09dc24-91f2-46f0-bb44-95576fc77236 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:57.994319 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09dc24-91f2-46f0-bb44-95576fc77236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:58.004576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-ba09dc24-91f2-46f0-bb44-95576fc77236 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:34:58.005851 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ba09dc24-91f2-46f0-bb44-95576fc77236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:58.067012 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1281/2534] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:34:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:34:58.076739 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a91cc1-23cb-439e-bc80-d19957920e09 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:58.078144 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-40a91cc1-23cb-439e-bc80-d19957920e09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:58.136231 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a91cc1-23cb-439e-bc80-d19957920e09 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:58.139846 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2bf170-92f1-414a-91f5-6b60f2d920eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:58.140779 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bf170-92f1-414a-91f5-6b60f2d920eb service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:58.141067 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bf170-92f1-414a-91f5-6b60f2d920eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:58.141353 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bf170-92f1-414a-91f5-6b60f2d920eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:58.151384 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2bf170-92f1-414a-91f5-6b60f2d920eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:58.152839 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2bf170-92f1-414a-91f5-6b60f2d920eb service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:58.162922 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1254/2535] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:34:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:34:58.658474 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09dc24-91f2-46f0-bb44-95576fc77236 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:58.661089 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ba09dc24-91f2-46f0-bb44-95576fc77236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:58.683189 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09dc24-91f2-46f0-bb44-95576fc77236 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:58.692752 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ec74c-bd7b-4616-beab-7160feb37d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:58.694249 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ec74c-bd7b-4616-beab-7160feb37d0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:58.694742 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ec74c-bd7b-4616-beab-7160feb37d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:58.695226 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ec74c-bd7b-4616-beab-7160feb37d0b admin admin] PATH_INFO: `/v3/projects/63378d84e46c4737b89aa861626f155c` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:58.697414 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564ec74c-bd7b-4616-beab-7160feb37d0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=63378d84e46c4737b89aa861626f155c)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:58.699947 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564ec74c-bd7b-4616-beab-7160feb37d0b admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:58.831552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-564ec74c-bd7b-4616-beab-7160feb37d0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63378d84e46c4737b89aa861626f155c', 'request_id': 'req-564ec74c-bd7b-4616-beab-7160feb37d0b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:58.837882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-564ec74c-bd7b-4616-beab-7160feb37d0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63378d84e46c4737b89aa861626f155c', 'request_id': 'req-564ec74c-bd7b-4616-beab-7160feb37d0b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:58.900756 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-564ec74c-bd7b-4616-beab-7160feb37d0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63378d84e46c4737b89aa861626f155c', 'request_id': 'req-564ec74c-bd7b-4616-beab-7160feb37d0b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:58.911756 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-564ec74c-bd7b-4616-beab-7160feb37d0b admin admin] The token cache is being invalidate because project 63378d84e46c4737b89aa861626f155c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:34:58.911962 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-564ec74c-bd7b-4616-beab-7160feb37d0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:58.912688 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-564ec74c-bd7b-4616-beab-7160feb37d0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:34:58.914286 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1282/2536] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:34:58 2022] DELETE /identity/v3/projects/63378d84e46c4737b89aa861626f155c => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:34:58.919182 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e4279-b575-4396-b203-4a2c80f70a4b None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:58.919680 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e4279-b575-4396-b203-4a2c80f70a4b None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:58.919787 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e4279-b575-4396-b203-4a2c80f70a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:58.937445 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-e09e4279-b575-4396-b203-4a2c80f70a4b None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:34:58.938650 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e09e4279-b575-4396-b203-4a2c80f70a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:58.956481 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:34:58.998337 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1255/2537] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:34:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:34:59.006765 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f13d453-cc86-4561-aafc-8ebaff32c8c6 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:59.007277 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f13d453-cc86-4561-aafc-8ebaff32c8c6 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:59.007475 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f13d453-cc86-4561-aafc-8ebaff32c8c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:59.026335 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-9f13d453-cc86-4561-aafc-8ebaff32c8c6 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:34:59.027554 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-9f13d453-cc86-4561-aafc-8ebaff32c8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:59.035746 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1283/2538] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:34:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:34:59.039976 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e4279-b575-4396-b203-4a2c80f70a4b None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:59.049692 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e4279-b575-4396-b203-4a2c80f70a4b None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:59.054622 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ad394-7c9b-4262-9da7-3dfbbc371e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:59.055292 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ad394-7c9b-4262-9da7-3dfbbc371e7d admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:59.055585 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ad394-7c9b-4262-9da7-3dfbbc371e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:59.055869 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ad394-7c9b-4262-9da7-3dfbbc371e7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:59.056654 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2ad394-7c9b-4262-9da7-3dfbbc371e7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:34:59.056963 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2ad394-7c9b-4262-9da7-3dfbbc371e7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:59.058590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2ad394-7c9b-4262-9da7-3dfbbc371e7d admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:59.071663 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1256/2539] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:34:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:34:59.075934 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f13d453-cc86-4561-aafc-8ebaff32c8c6 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:59.086022 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f13d453-cc86-4561-aafc-8ebaff32c8c6 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:59.090769 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb42024-dedc-429d-9120-6716716d6827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:59.091425 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb42024-dedc-429d-9120-6716716d6827 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:59.091722 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb42024-dedc-429d-9120-6716716d6827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:59.092006 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb42024-dedc-429d-9120-6716716d6827 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:59.092734 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb42024-dedc-429d-9120-6716716d6827 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:59.094636 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb42024-dedc-429d-9120-6716716d6827 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:59.183114 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1284/2540] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:34:59 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:34:59.187095 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ad394-7c9b-4262-9da7-3dfbbc371e7d admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:59.197574 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ad394-7c9b-4262-9da7-3dfbbc371e7d admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:59.255309 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2339bfaa-bf37-44b2-b897-b6ee95229381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:59.255980 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2339bfaa-bf37-44b2-b897-b6ee95229381 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:59.256285 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2339bfaa-bf37-44b2-b897-b6ee95229381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:59.256589 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2339bfaa-bf37-44b2-b897-b6ee95229381 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:59.257292 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2339bfaa-bf37-44b2-b897-b6ee95229381 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:59.259224 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2339bfaa-bf37-44b2-b897-b6ee95229381 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:59.381398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1257/2541] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:34:59 2022] POST /identity/v3/users => generated 287 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:34:59.385309 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb42024-dedc-429d-9120-6716716d6827 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:59.396233 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb42024-dedc-429d-9120-6716716d6827 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:59.400837 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b39f6c2-3bff-43fb-964d-cf18ca373946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:59.401496 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39f6c2-3bff-43fb-964d-cf18ca373946 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:59.401779 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39f6c2-3bff-43fb-964d-cf18ca373946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:59.402058 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39f6c2-3bff-43fb-964d-cf18ca373946 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:59.402792 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b39f6c2-3bff-43fb-964d-cf18ca373946 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:59.404390 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b39f6c2-3bff-43fb-964d-cf18ca373946 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:59.415522 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1285/2542] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:34:59.424850 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2339bfaa-bf37-44b2-b897-b6ee95229381 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:59.435250 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2339bfaa-bf37-44b2-b897-b6ee95229381 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:59.439885 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49720f64-eef2-49ab-812e-05b61c2d1711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:59.440579 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49720f64-eef2-49ab-812e-05b61c2d1711 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:59.440866 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49720f64-eef2-49ab-812e-05b61c2d1711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:59.441154 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49720f64-eef2-49ab-812e-05b61c2d1711 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:59.441923 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49720f64-eef2-49ab-812e-05b61c2d1711 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:59.443508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49720f64-eef2-49ab-812e-05b61c2d1711 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:59.453511 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1258/2543] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:34:59.458311 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b39f6c2-3bff-43fb-964d-cf18ca373946 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:59.468557 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b39f6c2-3bff-43fb-964d-cf18ca373946 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:59.473924 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707aa14d-e31f-41be-95b5-b4f27744c14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:59.474852 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707aa14d-e31f-41be-95b5-b4f27744c14d admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:59.475140 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707aa14d-e31f-41be-95b5-b4f27744c14d admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:59.475422 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707aa14d-e31f-41be-95b5-b4f27744c14d admin admin] PATH_INFO: `/v3/projects/a09c8e77be6a4cad9ac7079444c33b4e/users/0ed2a6ccd91b406c85eed229a3f5a186/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:59.490289 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707aa14d-e31f-41be-95b5-b4f27744c14d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a09c8e77be6a4cad9ac7079444c33b4e, user_id=0ed2a6ccd91b406c85eed229a3f5a186, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:59.493380 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707aa14d-e31f-41be-95b5-b4f27744c14d admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:59.570178 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1286/2544] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:34:59 2022] PUT /identity/v3/projects/a09c8e77be6a4cad9ac7079444c33b4e/users/0ed2a6ccd91b406c85eed229a3f5a186/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:34:59.580687 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364037a-80db-4ad3-8a5e-12e733e55bac None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:59.580986 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364037a-80db-4ad3-8a5e-12e733e55bac None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:59.581380 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364037a-80db-4ad3-8a5e-12e733e55bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:59.609658 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-e364037a-80db-4ad3-8a5e-12e733e55bac None None] MFA Rules not processed for user `0ed2a6ccd91b406c85eed229a3f5a186`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:34:59.611140 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e364037a-80db-4ad3-8a5e-12e733e55bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:59.679261 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1259/2545] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:34:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:34:59.690228 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707aa14d-e31f-41be-95b5-b4f27744c14d admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:34:59.692687 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-707aa14d-e31f-41be-95b5-b4f27744c14d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:34:59.715418 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:34:59.750960 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707aa14d-e31f-41be-95b5-b4f27744c14d admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:34:59.755804 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75073a18-501f-4ec3-80b6-d85afb6b592a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:34:59.756517 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75073a18-501f-4ec3-80b6-d85afb6b592a service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:34:59.756898 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75073a18-501f-4ec3-80b6-d85afb6b592a service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:34:59.757181 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75073a18-501f-4ec3-80b6-d85afb6b592a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:34:59.766821 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75073a18-501f-4ec3-80b6-d85afb6b592a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:34:59.768263 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75073a18-501f-4ec3-80b6-d85afb6b592a service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:34:59.779251 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1287/2546] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:34:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:35:04.918656 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364037a-80db-4ad3-8a5e-12e733e55bac None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:35:04.937834 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:35:04.984255 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364037a-80db-4ad3-8a5e-12e733e55bac None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:35:04.988798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d592bb6-c7d8-4d9e-b665-eed24cad55e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:35:04.989695 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d592bb6-c7d8-4d9e-b665-eed24cad55e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:04.990643 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d592bb6-c7d8-4d9e-b665-eed24cad55e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:04.990924 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d592bb6-c7d8-4d9e-b665-eed24cad55e2 admin admin] PATH_INFO: `/v3/users/0ed2a6ccd91b406c85eed229a3f5a186` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:04.992452 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d592bb6-c7d8-4d9e-b665-eed24cad55e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ed2a6ccd91b406c85eed229a3f5a186)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:35:04.994410 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d592bb6-c7d8-4d9e-b665-eed24cad55e2 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:35:05.148204 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9d592bb6-c7d8-4d9e-b665-eed24cad55e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ed2a6ccd91b406c85eed229a3f5a186'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:05.155195 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9d592bb6-c7d8-4d9e-b665-eed24cad55e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ed2a6ccd91b406c85eed229a3f5a186'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:05.168028 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9d592bb6-c7d8-4d9e-b665-eed24cad55e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ed2a6ccd91b406c85eed229a3f5a186'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:05.168734 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9d592bb6-c7d8-4d9e-b665-eed24cad55e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ed2a6ccd91b406c85eed229a3f5a186'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:05.169343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9d592bb6-c7d8-4d9e-b665-eed24cad55e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ed2a6ccd91b406c85eed229a3f5a186'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:05.264853 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1260/2547] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:35:04 2022] DELETE /identity/v3/users/0ed2a6ccd91b406c85eed229a3f5a186 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:35:05.269739 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15548739-3b11-469f-a74d-919a1790458d None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:05.270262 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15548739-3b11-469f-a74d-919a1790458d None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:05.270367 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15548739-3b11-469f-a74d-919a1790458d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:05.289262 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-15548739-3b11-469f-a74d-919a1790458d None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:35:05.290475 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-15548739-3b11-469f-a74d-919a1790458d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:35:05.349008 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1288/2548] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:35:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:35:05.359362 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d592bb6-c7d8-4d9e-b665-eed24cad55e2 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:35:05.360829 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-9d592bb6-c7d8-4d9e-b665-eed24cad55e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:35:05.418064 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d592bb6-c7d8-4d9e-b665-eed24cad55e2 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:35:05.421564 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633cf9f9-e95e-451c-8b55-caf328c12e3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:35:05.422203 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633cf9f9-e95e-451c-8b55-caf328c12e3a service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:05.422487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633cf9f9-e95e-451c-8b55-caf328c12e3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:05.422766 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633cf9f9-e95e-451c-8b55-caf328c12e3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:05.431672 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633cf9f9-e95e-451c-8b55-caf328c12e3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:35:05.433112 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633cf9f9-e95e-451c-8b55-caf328c12e3a service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:35:05.442455 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1261/2549] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:35:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:35:05.876822 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15548739-3b11-469f-a74d-919a1790458d None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:35:05.878342 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-15548739-3b11-469f-a74d-919a1790458d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:35:05.891606 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15548739-3b11-469f-a74d-919a1790458d None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:35:05.896955 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8416a543-0258-4377-be01-363440825455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:35:05.897841 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8416a543-0258-4377-be01-363440825455 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:05.900466 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8416a543-0258-4377-be01-363440825455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:05.900466 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8416a543-0258-4377-be01-363440825455 admin admin] PATH_INFO: `/v3/projects/a09c8e77be6a4cad9ac7079444c33b4e` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:05.900466 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8416a543-0258-4377-be01-363440825455 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a09c8e77be6a4cad9ac7079444c33b4e)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:35:05.901578 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8416a543-0258-4377-be01-363440825455 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:35:06.026327 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8416a543-0258-4377-be01-363440825455 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a09c8e77be6a4cad9ac7079444c33b4e', 'request_id': 'req-8416a543-0258-4377-be01-363440825455'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:06.032432 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8416a543-0258-4377-be01-363440825455 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a09c8e77be6a4cad9ac7079444c33b4e', 'request_id': 'req-8416a543-0258-4377-be01-363440825455'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:06.123430 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8416a543-0258-4377-be01-363440825455 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a09c8e77be6a4cad9ac7079444c33b4e', 'request_id': 'req-8416a543-0258-4377-be01-363440825455'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:06.135394 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8416a543-0258-4377-be01-363440825455 admin admin] The token cache is being invalidate because project a09c8e77be6a4cad9ac7079444c33b4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:35:06.135689 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8416a543-0258-4377-be01-363440825455 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:06.136373 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8416a543-0258-4377-be01-363440825455 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:06.137969 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1289/2550] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:35:05 2022] DELETE /identity/v3/projects/a09c8e77be6a4cad9ac7079444c33b4e => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:35:06.142839 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a481178-6941-4a65-ad31-ad7a2289a7d9 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:06.143341 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a481178-6941-4a65-ad31-ad7a2289a7d9 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:06.143441 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a481178-6941-4a65-ad31-ad7a2289a7d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:06.162593 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-4a481178-6941-4a65-ad31-ad7a2289a7d9 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:35:06.163806 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4a481178-6941-4a65-ad31-ad7a2289a7d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:35:06.198291 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:35:06.223544 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1262/2551] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:35:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:35:06.287688 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c08ce2-227b-406d-8098-ca86a92382be None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:06.288226 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c08ce2-227b-406d-8098-ca86a92382be None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:06.288226 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c08ce2-227b-406d-8098-ca86a92382be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:06.307283 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-81c08ce2-227b-406d-8098-ca86a92382be None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:35:06.308537 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-81c08ce2-227b-406d-8098-ca86a92382be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:35:06.316590 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1290/2552] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:35:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:35:06.320845 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a481178-6941-4a65-ad31-ad7a2289a7d9 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:35:06.330496 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a481178-6941-4a65-ad31-ad7a2289a7d9 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:35:06.335107 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c763c-8cc7-47c3-91cb-6ba38b54d85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:35:06.335758 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c763c-8cc7-47c3-91cb-6ba38b54d85a admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:06.336043 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c763c-8cc7-47c3-91cb-6ba38b54d85a admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:06.336325 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c763c-8cc7-47c3-91cb-6ba38b54d85a admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:06.337170 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4c763c-8cc7-47c3-91cb-6ba38b54d85a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:35:06.337480 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4c763c-8cc7-47c3-91cb-6ba38b54d85a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:35:06.339067 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4c763c-8cc7-47c3-91cb-6ba38b54d85a admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:35:06.351340 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1263/2553] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:35:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 06:35:06.355719 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c08ce2-227b-406d-8098-ca86a92382be None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:35:06.365738 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c08ce2-227b-406d-8098-ca86a92382be None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:35:06.371547 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd3a39c-4f07-4e4e-bb73-5e3224daef6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:35:06.372200 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3a39c-4f07-4e4e-bb73-5e3224daef6c admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:06.372533 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3a39c-4f07-4e4e-bb73-5e3224daef6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:06.372820 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3a39c-4f07-4e4e-bb73-5e3224daef6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:06.373524 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd3a39c-4f07-4e4e-bb73-5e3224daef6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:35:06.375433 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd3a39c-4f07-4e4e-bb73-5e3224daef6c admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:35:06.467936 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1291/2554] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:35:06 2022] POST /identity/v3/projects => generated 375 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:35:06.471992 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c763c-8cc7-47c3-91cb-6ba38b54d85a admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:35:06.482535 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c763c-8cc7-47c3-91cb-6ba38b54d85a admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:35:06.539927 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6f6a57-4c8a-4f5f-a0b9-e94f35b09a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:35:06.540553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f6a57-4c8a-4f5f-a0b9-e94f35b09a14 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:06.540848 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f6a57-4c8a-4f5f-a0b9-e94f35b09a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:06.541129 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f6a57-4c8a-4f5f-a0b9-e94f35b09a14 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:06.541833 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6f6a57-4c8a-4f5f-a0b9-e94f35b09a14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:35:06.543749 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6f6a57-4c8a-4f5f-a0b9-e94f35b09a14 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:35:06.621240 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1264/2555] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:35:06 2022] POST /identity/v3/users => generated 288 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:35:06.625379 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd3a39c-4f07-4e4e-bb73-5e3224daef6c admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:35:06.636786 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd3a39c-4f07-4e4e-bb73-5e3224daef6c admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:35:06.641685 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a46e9c6-0428-41ab-b640-e4105ba7ae15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:35:06.642359 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46e9c6-0428-41ab-b640-e4105ba7ae15 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:06.642653 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46e9c6-0428-41ab-b640-e4105ba7ae15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:06.642937 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46e9c6-0428-41ab-b640-e4105ba7ae15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:06.643679 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a46e9c6-0428-41ab-b640-e4105ba7ae15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:35:06.645338 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a46e9c6-0428-41ab-b640-e4105ba7ae15 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:35:06.657011 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1292/2556] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:35:06.667441 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6f6a57-4c8a-4f5f-a0b9-e94f35b09a14 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:35:06.677516 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6f6a57-4c8a-4f5f-a0b9-e94f35b09a14 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:35:06.682200 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69149781-4fb7-4416-9d10-c3e2779eb964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:35:06.682860 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69149781-4fb7-4416-9d10-c3e2779eb964 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:06.683148 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69149781-4fb7-4416-9d10-c3e2779eb964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:06.683427 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69149781-4fb7-4416-9d10-c3e2779eb964 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:06.684190 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69149781-4fb7-4416-9d10-c3e2779eb964 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:35:06.685813 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69149781-4fb7-4416-9d10-c3e2779eb964 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:35:06.696178 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1265/2557] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:35:06.700540 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a46e9c6-0428-41ab-b640-e4105ba7ae15 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:35:06.711645 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a46e9c6-0428-41ab-b640-e4105ba7ae15 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:35:06.716404 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4e6f6e-bcf7-43a6-993a-ee4785b949fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:35:06.717317 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e6f6e-bcf7-43a6-993a-ee4785b949fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:06.717606 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e6f6e-bcf7-43a6-993a-ee4785b949fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:06.717892 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e6f6e-bcf7-43a6-993a-ee4785b949fc admin admin] PATH_INFO: `/v3/projects/625cf1c8ad524097867f9151e19547f1/users/f026266e13654edaae3916a1fadf7565/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:06.720346 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:35:06.734066 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4e6f6e-bcf7-43a6-993a-ee4785b949fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=625cf1c8ad524097867f9151e19547f1, user_id=f026266e13654edaae3916a1fadf7565, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:35:06.737073 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4e6f6e-bcf7-43a6-993a-ee4785b949fc admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:35:06.829500 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1293/2558] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:35:06 2022] PUT /identity/v3/projects/625cf1c8ad524097867f9151e19547f1/users/f026266e13654edaae3916a1fadf7565/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:35:06.840023 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed73e0-5da0-463d-a5ee-949d041df2d7 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:06.840315 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed73e0-5da0-463d-a5ee-949d041df2d7 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:06.840627 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed73e0-5da0-463d-a5ee-949d041df2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:06.869830 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-8fed73e0-5da0-463d-a5ee-949d041df2d7 None None] MFA Rules not processed for user `f026266e13654edaae3916a1fadf7565`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:35:06.871100 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8fed73e0-5da0-463d-a5ee-949d041df2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:35:06.938676 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1266/2559] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:35:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:35:06.950918 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4e6f6e-bcf7-43a6-993a-ee4785b949fc admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:35:06.953552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-9f4e6f6e-bcf7-43a6-993a-ee4785b949fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:35:07.015293 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4e6f6e-bcf7-43a6-993a-ee4785b949fc admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:35:07.018999 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f251d-50cd-47f9-a6d5-81c556c918ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:35:07.019658 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f251d-50cd-47f9-a6d5-81c556c918ac service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:07.019944 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f251d-50cd-47f9-a6d5-81c556c918ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:07.020221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f251d-50cd-47f9-a6d5-81c556c918ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:07.029261 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f251d-50cd-47f9-a6d5-81c556c918ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:35:07.030694 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f251d-50cd-47f9-a6d5-81c556c918ac service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:35:07.040517 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1294/2560] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:35:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:35:11.448696 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed73e0-5da0-463d-a5ee-949d041df2d7 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:35:11.450076 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8fed73e0-5da0-463d-a5ee-949d041df2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:35:11.489415 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:35:11.519835 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed73e0-5da0-463d-a5ee-949d041df2d7 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:35:11.525654 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b4bbe-627b-4daa-a4ae-b624839752c0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3560534427544cdabfd0ba702a0a165', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:35:11.526314 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b4bbe-627b-4daa-a4ae-b624839752c0 service placement] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:11.526609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b4bbe-627b-4daa-a4ae-b624839752c0 service placement] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:11.526888 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b4bbe-627b-4daa-a4ae-b624839752c0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:11.529560 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-cd8b4bbe-627b-4daa-a4ae-b624839752c0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:35:11.542102 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8b4bbe-627b-4daa-a4ae-b624839752c0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:35:11.545313 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8b4bbe-627b-4daa-a4ae-b624839752c0 service placement] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:35:11.557477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1267/2561] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:35:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:35:17.035456 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=33,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:35:21.563796 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=16 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:35:21.881019 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f251d-50cd-47f9-a6d5-81c556c918ac service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:35:21.952493 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f251d-50cd-47f9-a6d5-81c556c918ac service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:35:21.956872 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619c675-5f47-42bb-9528-0a7000c0a11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:35:21.957746 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619c675-5f47-42bb-9528-0a7000c0a11e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:21.958033 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619c675-5f47-42bb-9528-0a7000c0a11e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:21.958311 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619c675-5f47-42bb-9528-0a7000c0a11e admin admin] PATH_INFO: `/v3/users/f026266e13654edaae3916a1fadf7565` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:21.959920 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0619c675-5f47-42bb-9528-0a7000c0a11e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f026266e13654edaae3916a1fadf7565)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:35:21.962951 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0619c675-5f47-42bb-9528-0a7000c0a11e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:35:22.174199 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0619c675-5f47-42bb-9528-0a7000c0a11e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f026266e13654edaae3916a1fadf7565'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:22.181212 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0619c675-5f47-42bb-9528-0a7000c0a11e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f026266e13654edaae3916a1fadf7565'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:22.181896 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0619c675-5f47-42bb-9528-0a7000c0a11e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f026266e13654edaae3916a1fadf7565'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:22.182500 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0619c675-5f47-42bb-9528-0a7000c0a11e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f026266e13654edaae3916a1fadf7565'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:22.194521 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0619c675-5f47-42bb-9528-0a7000c0a11e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f026266e13654edaae3916a1fadf7565'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:22.340467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1295/2562] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:35:21 2022] DELETE /identity/v3/users/f026266e13654edaae3916a1fadf7565 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:35:22.345284 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41198278-0762-4ba9-a5cb-d275056ca3c4 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:22.345585 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41198278-0762-4ba9-a5cb-d275056ca3c4 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:22.345903 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41198278-0762-4ba9-a5cb-d275056ca3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:22.364000 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-41198278-0762-4ba9-a5cb-d275056ca3c4 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:35:22.365258 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-41198278-0762-4ba9-a5cb-d275056ca3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:35:22.427130 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1268/2563] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:35:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:35:22.437486 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619c675-5f47-42bb-9528-0a7000c0a11e admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:35:22.438923 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-0619c675-5f47-42bb-9528-0a7000c0a11e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:35:22.498166 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619c675-5f47-42bb-9528-0a7000c0a11e admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:35:22.501791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef8f207-8e9a-48de-88c1-d109d345fe24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:35:22.502439 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8f207-8e9a-48de-88c1-d109d345fe24 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:22.502725 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8f207-8e9a-48de-88c1-d109d345fe24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:22.503029 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8f207-8e9a-48de-88c1-d109d345fe24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:22.513325 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef8f207-8e9a-48de-88c1-d109d345fe24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:35:22.516071 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef8f207-8e9a-48de-88c1-d109d345fe24 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:35:22.524813 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1296/2564] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:35:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:35:22.997538 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41198278-0762-4ba9-a5cb-d275056ca3c4 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:35:22.999123 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-41198278-0762-4ba9-a5cb-d275056ca3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:35:23.015692 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41198278-0762-4ba9-a5cb-d275056ca3c4 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:35:23.021579 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e6ae19-1b81-43b1-871d-556dbdc20bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:35:23.022665 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e6ae19-1b81-43b1-871d-556dbdc20bdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:23.023179 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e6ae19-1b81-43b1-871d-556dbdc20bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:23.024182 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e6ae19-1b81-43b1-871d-556dbdc20bdc admin admin] PATH_INFO: `/v3/projects/625cf1c8ad524097867f9151e19547f1` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:23.025811 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e6ae19-1b81-43b1-871d-556dbdc20bdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=625cf1c8ad524097867f9151e19547f1)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:35:23.028085 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e6ae19-1b81-43b1-871d-556dbdc20bdc admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:35:23.146884 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-40e6ae19-1b81-43b1-871d-556dbdc20bdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '625cf1c8ad524097867f9151e19547f1', 'request_id': 'req-40e6ae19-1b81-43b1-871d-556dbdc20bdc'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:23.153369 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-40e6ae19-1b81-43b1-871d-556dbdc20bdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '625cf1c8ad524097867f9151e19547f1', 'request_id': 'req-40e6ae19-1b81-43b1-871d-556dbdc20bdc'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:23.240370 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-40e6ae19-1b81-43b1-871d-556dbdc20bdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '625cf1c8ad524097867f9151e19547f1', 'request_id': 'req-40e6ae19-1b81-43b1-871d-556dbdc20bdc'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:23.252805 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-40e6ae19-1b81-43b1-871d-556dbdc20bdc admin admin] The token cache is being invalidate because project 625cf1c8ad524097867f9151e19547f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:35:23.255677 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-40e6ae19-1b81-43b1-871d-556dbdc20bdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:23.255677 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-40e6ae19-1b81-43b1-871d-556dbdc20bdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:23.255677 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1269/2565] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:35:22 2022] DELETE /identity/v3/projects/625cf1c8ad524097867f9151e19547f1 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:35:23.260530 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13c97f-f342-41cd-a4a8-128bd996d940 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:23.261013 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13c97f-f342-41cd-a4a8-128bd996d940 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:23.261109 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13c97f-f342-41cd-a4a8-128bd996d940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:23.281258 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-df13c97f-f342-41cd-a4a8-128bd996d940 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:35:23.282476 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-df13c97f-f342-41cd-a4a8-128bd996d940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:35:23.301508 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:35:23.348380 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1297/2566] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:35:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:35:23.358199 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8553f-4a74-40fa-87af-086e281e32ff None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:23.358502 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8553f-4a74-40fa-87af-086e281e32ff None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:23.358790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8553f-4a74-40fa-87af-086e281e32ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:23.378010 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-49d8553f-4a74-40fa-87af-086e281e32ff None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:35:23.381926 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-49d8553f-4a74-40fa-87af-086e281e32ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:35:23.388027 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1270/2567] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:35:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:35:23.392275 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df13c97f-f342-41cd-a4a8-128bd996d940 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:35:23.402704 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df13c97f-f342-41cd-a4a8-128bd996d940 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:35:23.407465 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a3d95-81cd-4069-bf90-a4a42684973f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:35:23.408127 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a3d95-81cd-4069-bf90-a4a42684973f admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:23.408443 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a3d95-81cd-4069-bf90-a4a42684973f admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:23.408725 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a3d95-81cd-4069-bf90-a4a42684973f admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:23.409483 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a3d95-81cd-4069-bf90-a4a42684973f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:35:23.409794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a3d95-81cd-4069-bf90-a4a42684973f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:35:23.411460 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a3d95-81cd-4069-bf90-a4a42684973f admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:35:23.423167 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1298/2568] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:35:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:35:23.427633 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8553f-4a74-40fa-87af-086e281e32ff None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:35:23.437226 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8553f-4a74-40fa-87af-086e281e32ff None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:35:23.442146 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eb7dd1-5bf0-40e5-8267-fd623cdb663f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:35:23.442811 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb7dd1-5bf0-40e5-8267-fd623cdb663f admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:23.443100 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb7dd1-5bf0-40e5-8267-fd623cdb663f admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:23.443405 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb7dd1-5bf0-40e5-8267-fd623cdb663f admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:23.444089 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eb7dd1-5bf0-40e5-8267-fd623cdb663f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:35:23.446657 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eb7dd1-5bf0-40e5-8267-fd623cdb663f admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:35:23.568602 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1271/2569] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:35:23 2022] POST /identity/v3/projects => generated 381 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 06:35:23.573688 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a3d95-81cd-4069-bf90-a4a42684973f admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:35:23.584217 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a3d95-81cd-4069-bf90-a4a42684973f admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:35:23.645509 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e8e06-9a40-4708-8817-7978f41b3dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:35:23.646226 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e8e06-9a40-4708-8817-7978f41b3dce admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:23.646515 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e8e06-9a40-4708-8817-7978f41b3dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:23.646816 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e8e06-9a40-4708-8817-7978f41b3dce admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:23.647509 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4e8e06-9a40-4708-8817-7978f41b3dce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:35:23.650240 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4e8e06-9a40-4708-8817-7978f41b3dce admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:35:23.735951 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1299/2570] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:35:23 2022] POST /identity/v3/users => generated 291 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:35:23.737258 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eb7dd1-5bf0-40e5-8267-fd623cdb663f admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:35:23.748316 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eb7dd1-5bf0-40e5-8267-fd623cdb663f admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:35:23.754135 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e2da4-4fda-45dc-b1ae-cde9f1b4ddff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:35:23.754811 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e2da4-4fda-45dc-b1ae-cde9f1b4ddff admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:23.755103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e2da4-4fda-45dc-b1ae-cde9f1b4ddff admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:23.755388 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e2da4-4fda-45dc-b1ae-cde9f1b4ddff admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:23.756180 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94e2da4-4fda-45dc-b1ae-cde9f1b4ddff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:35:23.759388 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94e2da4-4fda-45dc-b1ae-cde9f1b4ddff admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:35:23.774767 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1272/2571] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:35:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:35:23.776018 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e8e06-9a40-4708-8817-7978f41b3dce admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:35:23.786419 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e8e06-9a40-4708-8817-7978f41b3dce admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:35:23.791435 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4a3cca-b272-4f39-b39c-d91ecab4c54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:35:23.792118 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a3cca-b272-4f39-b39c-d91ecab4c54c admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:23.792520 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a3cca-b272-4f39-b39c-d91ecab4c54c admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:23.792815 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a3cca-b272-4f39-b39c-d91ecab4c54c admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:23.793547 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4a3cca-b272-4f39-b39c-d91ecab4c54c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:35:23.795153 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4a3cca-b272-4f39-b39c-d91ecab4c54c admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:35:23.807374 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1300/2572] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:35:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:35:23.811602 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e2da4-4fda-45dc-b1ae-cde9f1b4ddff admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:35:23.822795 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e2da4-4fda-45dc-b1ae-cde9f1b4ddff admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:35:23.827504 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a94e09-efc5-4db5-aa2a-3aaa51cfc599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:35:23.828386 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a94e09-efc5-4db5-aa2a-3aaa51cfc599 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:23.828704 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a94e09-efc5-4db5-aa2a-3aaa51cfc599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:23.828984 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a94e09-efc5-4db5-aa2a-3aaa51cfc599 admin admin] PATH_INFO: `/v3/projects/8a5093b9dbae43a3bf51e35f3825e8e4/users/de31ee0904954d55b877e173ddc2c8be/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:23.846132 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a94e09-efc5-4db5-aa2a-3aaa51cfc599 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a5093b9dbae43a3bf51e35f3825e8e4, user_id=de31ee0904954d55b877e173ddc2c8be, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:35:23.847582 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a94e09-efc5-4db5-aa2a-3aaa51cfc599 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:35:23.938409 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1273/2573] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:35:23 2022] PUT /identity/v3/projects/8a5093b9dbae43a3bf51e35f3825e8e4/users/de31ee0904954d55b877e173ddc2c8be/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:35:23.943504 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad92d5-e261-475e-a775-0428fe86eb62 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:23.943818 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad92d5-e261-475e-a775-0428fe86eb62 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:23.944106 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad92d5-e261-475e-a775-0428fe86eb62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:23.963089 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-dfad92d5-e261-475e-a775-0428fe86eb62 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:35:23.964450 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-dfad92d5-e261-475e-a775-0428fe86eb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:35:24.024905 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1301/2574] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:35:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:35:24.034406 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a94e09-efc5-4db5-aa2a-3aaa51cfc599 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:35:24.036165 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-46a94e09-efc5-4db5-aa2a-3aaa51cfc599 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:35:24.058673 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:35:24.093538 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a94e09-efc5-4db5-aa2a-3aaa51cfc599 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:35:24.097177 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e19b59-dec3-4456-a8a3-b6446aa5be59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:35:24.097821 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e19b59-dec3-4456-a8a3-b6446aa5be59 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:24.100689 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e19b59-dec3-4456-a8a3-b6446aa5be59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:24.100689 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e19b59-dec3-4456-a8a3-b6446aa5be59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:24.107857 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e19b59-dec3-4456-a8a3-b6446aa5be59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:35:24.109302 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e19b59-dec3-4456-a8a3-b6446aa5be59 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:35:24.119583 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1274/2575] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:35:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:35:25.467275 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a75ac-54e8-423a-914c-091e363ce506 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:25.471162 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a75ac-54e8-423a-914c-091e363ce506 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:25.471162 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a75ac-54e8-423a-914c-091e363ce506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:25.498986 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-6d8a75ac-54e8-423a-914c-091e363ce506 None None] MFA Rules not processed for user `de31ee0904954d55b877e173ddc2c8be`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:35:25.500572 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-6d8a75ac-54e8-423a-914c-091e363ce506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:35:25.524181 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:35:25.589375 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1302/2576] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:35:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:35:25.602399 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e19b59-dec3-4456-a8a3-b6446aa5be59 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:35:25.603805 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-41e19b59-dec3-4456-a8a3-b6446aa5be59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:35:25.664709 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e19b59-dec3-4456-a8a3-b6446aa5be59 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:35:25.668257 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ce14f-b641-404a-9000-539930bcfe5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:35:25.668969 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ce14f-b641-404a-9000-539930bcfe5d service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:25.669257 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ce14f-b641-404a-9000-539930bcfe5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:25.669535 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ce14f-b641-404a-9000-539930bcfe5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:25.678080 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813ce14f-b641-404a-9000-539930bcfe5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:35:25.679497 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813ce14f-b641-404a-9000-539930bcfe5d service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:35:25.689356 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1275/2577] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:35:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:35:28.482216 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a2c80-0489-4154-b69b-cd4612ff195b None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:28.482652 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a2c80-0489-4154-b69b-cd4612ff195b None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:28.482652 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a2c80-0489-4154-b69b-cd4612ff195b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:28.501249 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-5f0a2c80-0489-4154-b69b-cd4612ff195b None None] MFA Rules not processed for user `de31ee0904954d55b877e173ddc2c8be`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:35:28.504477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-5f0a2c80-0489-4154-b69b-cd4612ff195b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:35:28.509750 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1303/2578] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:35:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:35:28.612856 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d9cc5-6429-498e-81f6-ddf0005bc674 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:28.613376 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d9cc5-6429-498e-81f6-ddf0005bc674 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:28.613463 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d9cc5-6429-498e-81f6-ddf0005bc674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:28.647905 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-413d9cc5-6429-498e-81f6-ddf0005bc674 None None] MFA Rules not processed for user `b84dde57ff8e47419866377a819bea21`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:35:28.649908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-413d9cc5-6429-498e-81f6-ddf0005bc674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:35:28.718671 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1276/2579] 10.222.0.40 () {58 vars in 967 bytes} [Mon Jul 18 06:35:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:35:28.886416 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0a2c80-0489-4154-b69b-cd4612ff195b None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:35:28.888064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-5f0a2c80-0489-4154-b69b-cd4612ff195b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:35:28.964948 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0a2c80-0489-4154-b69b-cd4612ff195b None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:35:28.970277 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048de570-6bfe-450e-9a89-d68104e8f64d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0681f5fbea6d415c9ad2f3afae256a5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:35:28.970876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048de570-6bfe-450e-9a89-d68104e8f64d service swift] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:28.971165 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048de570-6bfe-450e-9a89-d68104e8f64d service swift] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:28.971444 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048de570-6bfe-450e-9a89-d68104e8f64d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:28.981661 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048de570-6bfe-450e-9a89-d68104e8f64d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:35:28.983297 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048de570-6bfe-450e-9a89-d68104e8f64d service swift] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:35:28.993897 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1304/2580] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Jul 18 06:35:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 06:35:35.145453 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d9cc5-6429-498e-81f6-ddf0005bc674 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:35:35.146896 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-413d9cc5-6429-498e-81f6-ddf0005bc674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:35:35.222600 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:35:35.228609 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d9cc5-6429-498e-81f6-ddf0005bc674 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:35:35.234054 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b030a63-b12b-4c2b-ac79-5af82b8ccffb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3560534427544cdabfd0ba702a0a165', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:35:35.234864 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b030a63-b12b-4c2b-ac79-5af82b8ccffb service placement] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:35.235156 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b030a63-b12b-4c2b-ac79-5af82b8ccffb service placement] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:35.235441 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b030a63-b12b-4c2b-ac79-5af82b8ccffb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:35.237491 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-0b030a63-b12b-4c2b-ac79-5af82b8ccffb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:35:35.252274 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b030a63-b12b-4c2b-ac79-5af82b8ccffb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:35:35.254615 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b030a63-b12b-4c2b-ac79-5af82b8ccffb service placement] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:35:35.269905 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1277/2581] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:35:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:35:38.989463 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=54 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:35:39.006387 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048de570-6bfe-450e-9a89-d68104e8f64d service swift] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:35:39.009037 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-048de570-6bfe-450e-9a89-d68104e8f64d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:35:39.027054 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048de570-6bfe-450e-9a89-d68104e8f64d service swift] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:35:39.031699 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d4b62f-6f0a-48e6-8f3c-84912c1b9b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:35:39.032617 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4b62f-6f0a-48e6-8f3c-84912c1b9b03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:39.032913 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4b62f-6f0a-48e6-8f3c-84912c1b9b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:39.033196 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4b62f-6f0a-48e6-8f3c-84912c1b9b03 admin admin] PATH_INFO: `/v3/users/81d9b70897ad4f8fabe72d1728558181` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:39.034404 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d4b62f-6f0a-48e6-8f3c-84912c1b9b03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81d9b70897ad4f8fabe72d1728558181)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:35:39.036334 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d4b62f-6f0a-48e6-8f3c-84912c1b9b03 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:35:39.199418 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-63d4b62f-6f0a-48e6-8f3c-84912c1b9b03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81d9b70897ad4f8fabe72d1728558181'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:39.206116 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-63d4b62f-6f0a-48e6-8f3c-84912c1b9b03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81d9b70897ad4f8fabe72d1728558181'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:39.206775 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-63d4b62f-6f0a-48e6-8f3c-84912c1b9b03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81d9b70897ad4f8fabe72d1728558181'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:39.207376 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-63d4b62f-6f0a-48e6-8f3c-84912c1b9b03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81d9b70897ad4f8fabe72d1728558181'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:39.219309 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-63d4b62f-6f0a-48e6-8f3c-84912c1b9b03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81d9b70897ad4f8fabe72d1728558181'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:39.280868 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1305/2582] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:35:39 2022] DELETE /identity/v3/users/81d9b70897ad4f8fabe72d1728558181 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:35:39.546375 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b030a63-b12b-4c2b-ac79-5af82b8ccffb service placement] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:35:39.557455 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b030a63-b12b-4c2b-ac79-5af82b8ccffb service placement] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:35:39.615530 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3148c45d-d7a0-4d07-955f-54037a6049ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:35:39.616449 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3148c45d-d7a0-4d07-955f-54037a6049ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:39.616993 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3148c45d-d7a0-4d07-955f-54037a6049ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:39.617298 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3148c45d-d7a0-4d07-955f-54037a6049ed admin admin] PATH_INFO: `/v3/projects/c7a3d0bcdbd24e498ab3664983af9298` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:39.618741 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3148c45d-d7a0-4d07-955f-54037a6049ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7a3d0bcdbd24e498ab3664983af9298)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:35:39.620717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3148c45d-d7a0-4d07-955f-54037a6049ed admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:35:39.716132 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3148c45d-d7a0-4d07-955f-54037a6049ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7a3d0bcdbd24e498ab3664983af9298', 'request_id': 'req-3148c45d-d7a0-4d07-955f-54037a6049ed'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:39.721977 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3148c45d-d7a0-4d07-955f-54037a6049ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7a3d0bcdbd24e498ab3664983af9298', 'request_id': 'req-3148c45d-d7a0-4d07-955f-54037a6049ed'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:39.800129 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3148c45d-d7a0-4d07-955f-54037a6049ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7a3d0bcdbd24e498ab3664983af9298', 'request_id': 'req-3148c45d-d7a0-4d07-955f-54037a6049ed'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:39.811967 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3148c45d-d7a0-4d07-955f-54037a6049ed admin admin] The token cache is being invalidate because project c7a3d0bcdbd24e498ab3664983af9298 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:35:39.812264 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3148c45d-d7a0-4d07-955f-54037a6049ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:39.812943 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3148c45d-d7a0-4d07-955f-54037a6049ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:39.814495 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1278/2583] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:35:39 2022] DELETE /identity/v3/projects/c7a3d0bcdbd24e498ab3664983af9298 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:35:39.818719 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d4b62f-6f0a-48e6-8f3c-84912c1b9b03 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:35:39.820171 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-63d4b62f-6f0a-48e6-8f3c-84912c1b9b03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:35:39.833233 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d4b62f-6f0a-48e6-8f3c-84912c1b9b03 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:35:39.889825 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da11d48a-a8cd-4568-8157-dd8d07eff2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:35:39.890705 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11d48a-a8cd-4568-8157-dd8d07eff2e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:39.890992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11d48a-a8cd-4568-8157-dd8d07eff2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:39.891267 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11d48a-a8cd-4568-8157-dd8d07eff2e8 admin admin] PATH_INFO: `/v3/users/d29d4b602518487cb496bb26c6bf9780` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:39.892478 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da11d48a-a8cd-4568-8157-dd8d07eff2e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d29d4b602518487cb496bb26c6bf9780)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:35:39.894402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da11d48a-a8cd-4568-8157-dd8d07eff2e8 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:35:40.075394 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-da11d48a-a8cd-4568-8157-dd8d07eff2e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd29d4b602518487cb496bb26c6bf9780'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:40.087779 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-da11d48a-a8cd-4568-8157-dd8d07eff2e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd29d4b602518487cb496bb26c6bf9780'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:40.089056 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-da11d48a-a8cd-4568-8157-dd8d07eff2e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd29d4b602518487cb496bb26c6bf9780'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:40.090315 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-da11d48a-a8cd-4568-8157-dd8d07eff2e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd29d4b602518487cb496bb26c6bf9780'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:40.105864 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-da11d48a-a8cd-4568-8157-dd8d07eff2e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd29d4b602518487cb496bb26c6bf9780'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:40.107590 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:35:40.177471 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1306/2584] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:35:39 2022] DELETE /identity/v3/users/d29d4b602518487cb496bb26c6bf9780 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:35:40.433732 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3148c45d-d7a0-4d07-955f-54037a6049ed admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:35:40.435293 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-3148c45d-d7a0-4d07-955f-54037a6049ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:35:40.450383 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3148c45d-d7a0-4d07-955f-54037a6049ed admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:35:40.512073 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd8ae7f-86b5-4eb5-8553-b378cd9ee6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:35:40.513114 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8ae7f-86b5-4eb5-8553-b378cd9ee6d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:40.513403 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8ae7f-86b5-4eb5-8553-b378cd9ee6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:40.513738 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8ae7f-86b5-4eb5-8553-b378cd9ee6d0 admin admin] PATH_INFO: `/v3/projects/0e44d69624be4720b0151f6cd16cde34` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:40.514971 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd8ae7f-86b5-4eb5-8553-b378cd9ee6d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e44d69624be4720b0151f6cd16cde34)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:35:40.516976 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd8ae7f-86b5-4eb5-8553-b378cd9ee6d0 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:35:40.542379 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:35:40.623923 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3bd8ae7f-86b5-4eb5-8553-b378cd9ee6d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e44d69624be4720b0151f6cd16cde34', 'request_id': 'req-3bd8ae7f-86b5-4eb5-8553-b378cd9ee6d0'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:40.629412 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3bd8ae7f-86b5-4eb5-8553-b378cd9ee6d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e44d69624be4720b0151f6cd16cde34', 'request_id': 'req-3bd8ae7f-86b5-4eb5-8553-b378cd9ee6d0'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:40.691411 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3bd8ae7f-86b5-4eb5-8553-b378cd9ee6d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e44d69624be4720b0151f6cd16cde34', 'request_id': 'req-3bd8ae7f-86b5-4eb5-8553-b378cd9ee6d0'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:40.704125 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3bd8ae7f-86b5-4eb5-8553-b378cd9ee6d0 admin admin] The token cache is being invalidate because project 0e44d69624be4720b0151f6cd16cde34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:35:40.704431 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3bd8ae7f-86b5-4eb5-8553-b378cd9ee6d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:40.705092 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3bd8ae7f-86b5-4eb5-8553-b378cd9ee6d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:35:40.706637 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1279/2585] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:35:40 2022] DELETE /identity/v3/projects/0e44d69624be4720b0151f6cd16cde34 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:35:50.400101 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:35:50.800095 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=15,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:35:51.944441 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da11d48a-a8cd-4568-8157-dd8d07eff2e8 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:35:51.945856 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-da11d48a-a8cd-4568-8157-dd8d07eff2e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:35:52.005739 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da11d48a-a8cd-4568-8157-dd8d07eff2e8 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:35:52.059000 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d55a81-c459-45ca-8606-6900cdba8116 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:35:52.059667 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d55a81-c459-45ca-8606-6900cdba8116 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:35:52.059954 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d55a81-c459-45ca-8606-6900cdba8116 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:35:52.060240 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d55a81-c459-45ca-8606-6900cdba8116 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:35:52.062721 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-75d55a81-c459-45ca-8606-6900cdba8116 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:35:52.123563 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d55a81-c459-45ca-8606-6900cdba8116 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:35:52.125049 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d55a81-c459-45ca-8606-6900cdba8116 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:35:52.134121 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1307/2586] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:35:51 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:36:02.129915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=68 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:36:10.031376 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dfd909-d0f8-44e7-8162-bb622bb488cf None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:36:10.034393 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dfd909-d0f8-44e7-8162-bb622bb488cf None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:36:10.034393 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dfd909-d0f8-44e7-8162-bb622bb488cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:36:10.049747 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-f4dfd909-d0f8-44e7-8162-bb622bb488cf None None] MFA Rules not processed for user `621a7ea4f60545d5a5c30b2b7a370bf0`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:36:10.050956 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f4dfd909-d0f8-44e7-8162-bb622bb488cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:36:10.057917 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1280/2587] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:36:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:36:10.094147 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d55a81-c459-45ca-8606-6900cdba8116 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:36:10.095572 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-75d55a81-c459-45ca-8606-6900cdba8116 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:36:10.106539 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d55a81-c459-45ca-8606-6900cdba8116 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:36:10.110236 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5ac1f6-e777-4e7e-a126-cc33ba73fc78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:36:10.110885 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ac1f6-e777-4e7e-a126-cc33ba73fc78 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:36:10.111168 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ac1f6-e777-4e7e-a126-cc33ba73fc78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:36:10.111447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ac1f6-e777-4e7e-a126-cc33ba73fc78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:36:10.113213 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-cc5ac1f6-e777-4e7e-a126-cc33ba73fc78 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:36:10.185236 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5ac1f6-e777-4e7e-a126-cc33ba73fc78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:36:10.186683 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5ac1f6-e777-4e7e-a126-cc33ba73fc78 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:36:10.198345 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1308/2588] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:36:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:36:11.506489 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dfd909-d0f8-44e7-8162-bb622bb488cf None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:36:11.508139 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f4dfd909-d0f8-44e7-8162-bb622bb488cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:36:11.612539 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dfd909-d0f8-44e7-8162-bb622bb488cf None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:36:11.634236 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2e10f-15b9-4f72-9b72-166e8a0c1497 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:36:11.634912 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2e10f-15b9-4f72-9b72-166e8a0c1497 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:36:11.635202 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2e10f-15b9-4f72-9b72-166e8a0c1497 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:36:11.635480 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2e10f-15b9-4f72-9b72-166e8a0c1497 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:36:11.639313 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b3c2e10f-15b9-4f72-9b72-166e8a0c1497 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:36:11.666010 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c2e10f-15b9-4f72-9b72-166e8a0c1497 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:36:11.667463 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c2e10f-15b9-4f72-9b72-166e8a0c1497 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:36:11.679416 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1281/2589] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:36:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:36:20.193298 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=34 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:36:21.673818 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=29 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:36:48.853380 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec69296-38f0-4036-beca-2c413eed47bb None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:36:48.853911 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec69296-38f0-4036-beca-2c413eed47bb None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:36:48.854009 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec69296-38f0-4036-beca-2c413eed47bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:36:48.907869 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-dec69296-38f0-4036-beca-2c413eed47bb None None] MFA Rules not processed for user `b84dde57ff8e47419866377a819bea21`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:36:48.909143 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-dec69296-38f0-4036-beca-2c413eed47bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:36:49.005073 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1309/2590] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Jul 18 06:36:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:36:49.171329 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2e10f-15b9-4f72-9b72-166e8a0c1497 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:36:49.172881 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b3c2e10f-15b9-4f72-9b72-166e8a0c1497 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:36:49.271057 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2e10f-15b9-4f72-9b72-166e8a0c1497 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:36:49.276456 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e3309-eed8-4574-96bf-98292d773d25 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0681f5fbea6d415c9ad2f3afae256a5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:36:49.277126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e3309-eed8-4574-96bf-98292d773d25 service swift] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:36:49.277411 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e3309-eed8-4574-96bf-98292d773d25 service swift] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:36:49.277690 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e3309-eed8-4574-96bf-98292d773d25 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:36:49.288942 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1e3309-eed8-4574-96bf-98292d773d25 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:36:49.290857 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1e3309-eed8-4574-96bf-98292d773d25 service swift] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:36:49.300735 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1282/2591] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Jul 18 06:36:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 06:36:58.999432 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=40 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:36:59.296985 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=41 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:37:25.877228 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec69296-38f0-4036-beca-2c413eed47bb None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:37:25.887385 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec69296-38f0-4036-beca-2c413eed47bb None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:37:25.890958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743830a8-64b9-44c3-a224-e0be7a73affb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:37:25.891599 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743830a8-64b9-44c3-a224-e0be7a73affb service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:37:25.891887 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743830a8-64b9-44c3-a224-e0be7a73affb service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:37:25.892186 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743830a8-64b9-44c3-a224-e0be7a73affb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:37:25.893997 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-743830a8-64b9-44c3-a224-e0be7a73affb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:37:25.975385 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743830a8-64b9-44c3-a224-e0be7a73affb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:37:25.976876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743830a8-64b9-44c3-a224-e0be7a73affb service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:37:25.995460 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1310/2592] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:37:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:37:35.992821 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=41 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:37:38.732526 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e3309-eed8-4574-96bf-98292d773d25 service swift] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:37:38.743057 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e3309-eed8-4574-96bf-98292d773d25 service swift] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:37:38.746708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38190db1-02ed-4894-b97a-90794206b89d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:37:38.747341 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38190db1-02ed-4894-b97a-90794206b89d service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:37:38.747631 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38190db1-02ed-4894-b97a-90794206b89d service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:37:38.747910 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38190db1-02ed-4894-b97a-90794206b89d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:37:38.749719 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-38190db1-02ed-4894-b97a-90794206b89d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:37:38.761920 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38190db1-02ed-4894-b97a-90794206b89d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:37:38.763312 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38190db1-02ed-4894-b97a-90794206b89d service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:37:38.772216 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1283/2593] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:37:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:37:39.314519 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743830a8-64b9-44c3-a224-e0be7a73affb service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:37:39.324221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743830a8-64b9-44c3-a224-e0be7a73affb service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:37:39.327788 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1cf4fe-ab38-4db3-8dbe-d07d7a045474 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:37:39.328425 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1cf4fe-ab38-4db3-8dbe-d07d7a045474 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:37:39.328722 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1cf4fe-ab38-4db3-8dbe-d07d7a045474 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:37:39.328997 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1cf4fe-ab38-4db3-8dbe-d07d7a045474 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:37:39.330769 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-bb1cf4fe-ab38-4db3-8dbe-d07d7a045474 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:37:39.396746 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1cf4fe-ab38-4db3-8dbe-d07d7a045474 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:37:39.398226 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1cf4fe-ab38-4db3-8dbe-d07d7a045474 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:37:39.407972 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1311/2594] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:37:39 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:37:42.384511 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38190db1-02ed-4894-b97a-90794206b89d service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:37:42.385959 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-38190db1-02ed-4894-b97a-90794206b89d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:37:42.404265 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38190db1-02ed-4894-b97a-90794206b89d service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:37:42.410019 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f2343-e98e-4495-8d01-b1a574b7b3e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:37:42.411253 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f2343-e98e-4495-8d01-b1a574b7b3e4 service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:37:42.411563 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f2343-e98e-4495-8d01-b1a574b7b3e4 service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:37:42.411847 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f2343-e98e-4495-8d01-b1a574b7b3e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:37:42.413815 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-453f2343-e98e-4495-8d01-b1a574b7b3e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:37:42.427394 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453f2343-e98e-4495-8d01-b1a574b7b3e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:37:42.429543 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453f2343-e98e-4495-8d01-b1a574b7b3e4 service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:37:42.442669 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1284/2595] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:37:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:37:44.849834 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1cf4fe-ab38-4db3-8dbe-d07d7a045474 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:37:44.864804 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1cf4fe-ab38-4db3-8dbe-d07d7a045474 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:37:44.870507 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3be1c-fb8e-4bf8-8ed0-0e498d2994a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:37:44.871191 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3be1c-fb8e-4bf8-8ed0-0e498d2994a5 service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:37:44.871478 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3be1c-fb8e-4bf8-8ed0-0e498d2994a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:37:44.871756 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3be1c-fb8e-4bf8-8ed0-0e498d2994a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:37:44.873644 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-61f3be1c-fb8e-4bf8-8ed0-0e498d2994a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:37:44.893667 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f3be1c-fb8e-4bf8-8ed0-0e498d2994a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:37:44.895238 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f3be1c-fb8e-4bf8-8ed0-0e498d2994a5 service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:37:44.907650 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1312/2596] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:37:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:37:47.259266 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f2343-e98e-4495-8d01-b1a574b7b3e4 service nova] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:37:47.260799 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-453f2343-e98e-4495-8d01-b1a574b7b3e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:37:47.347125 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f2343-e98e-4495-8d01-b1a574b7b3e4 service nova] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:37:47.351763 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5c6cfa-f928-48d0-be64-a5628851ce59 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1193330cb5644765a7abb77738929ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:37:47.352437 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c6cfa-f928-48d0-be64-a5628851ce59 service glance] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:37:47.352763 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c6cfa-f928-48d0-be64-a5628851ce59 service glance] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:37:47.353094 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c6cfa-f928-48d0-be64-a5628851ce59 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:37:47.354918 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-0f5c6cfa-f928-48d0-be64-a5628851ce59 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:37:47.369425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5c6cfa-f928-48d0-be64-a5628851ce59 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:37:47.370847 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5c6cfa-f928-48d0-be64-a5628851ce59 service glance] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:37:47.381849 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1285/2597] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:37:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 06:37:47.620568 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3be1c-fb8e-4bf8-8ed0-0e498d2994a5 service nova] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:37:47.622005 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-61f3be1c-fb8e-4bf8-8ed0-0e498d2994a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:37:47.634671 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3be1c-fb8e-4bf8-8ed0-0e498d2994a5 service nova] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:37:47.638381 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803298ad-6366-4223-8022-3dd3b52f3241 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:37:47.639116 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803298ad-6366-4223-8022-3dd3b52f3241 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:37:47.639415 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803298ad-6366-4223-8022-3dd3b52f3241 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:37:47.639696 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803298ad-6366-4223-8022-3dd3b52f3241 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:37:47.641465 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-803298ad-6366-4223-8022-3dd3b52f3241 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:37:47.652247 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803298ad-6366-4223-8022-3dd3b52f3241 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:37:47.653789 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803298ad-6366-4223-8022-3dd3b52f3241 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:37:47.662636 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1313/2598] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:37:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:37:47.736739 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5c6cfa-f928-48d0-be64-a5628851ce59 service glance] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:37:47.746710 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5c6cfa-f928-48d0-be64-a5628851ce59 service glance] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:37:47.750324 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0211d58b-2b4f-4855-b58f-c3f30884a439 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:37:47.750960 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0211d58b-2b4f-4855-b58f-c3f30884a439 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:37:47.751249 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0211d58b-2b4f-4855-b58f-c3f30884a439 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:37:47.751529 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0211d58b-2b4f-4855-b58f-c3f30884a439 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:37:47.753412 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-0211d58b-2b4f-4855-b58f-c3f30884a439 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:37:47.767426 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0211d58b-2b4f-4855-b58f-c3f30884a439 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:37:47.768889 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0211d58b-2b4f-4855-b58f-c3f30884a439 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:37:47.780493 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1286/2599] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:37:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:37:47.843884 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803298ad-6366-4223-8022-3dd3b52f3241 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:37:47.845506 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-803298ad-6366-4223-8022-3dd3b52f3241 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:37:47.917388 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803298ad-6366-4223-8022-3dd3b52f3241 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:37:47.922576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa14b1-89ac-438d-bb0a-e214e8994435 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3560534427544cdabfd0ba702a0a165', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:37:47.923230 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa14b1-89ac-438d-bb0a-e214e8994435 service placement] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:37:47.923520 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa14b1-89ac-438d-bb0a-e214e8994435 service placement] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:37:47.923802 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa14b1-89ac-438d-bb0a-e214e8994435 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:37:47.925648 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-38aa14b1-89ac-438d-bb0a-e214e8994435 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:37:47.937508 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aa14b1-89ac-438d-bb0a-e214e8994435 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:37:47.939096 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aa14b1-89ac-438d-bb0a-e214e8994435 service placement] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:37:47.949304 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1314/2600] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:37:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:37:51.216300 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d73027-46db-47bf-82fc-71d2eaa8db1f None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:37:51.216794 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d73027-46db-47bf-82fc-71d2eaa8db1f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:37:51.216902 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d73027-46db-47bf-82fc-71d2eaa8db1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:37:51.234946 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-c5d73027-46db-47bf-82fc-71d2eaa8db1f None None] MFA Rules not processed for user `b84dde57ff8e47419866377a819bea21`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:37:51.236164 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-c5d73027-46db-47bf-82fc-71d2eaa8db1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:37:51.244925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1287/2601] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Jul 18 06:37:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:37:51.408001 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa14b1-89ac-438d-bb0a-e214e8994435 service placement] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:37:51.420266 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa14b1-89ac-438d-bb0a-e214e8994435 service placement] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:37:51.425509 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f3d56-6cef-4db6-817a-48bedd4e0664 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0681f5fbea6d415c9ad2f3afae256a5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:37:51.426160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f3d56-6cef-4db6-817a-48bedd4e0664 service swift] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:37:51.426454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f3d56-6cef-4db6-817a-48bedd4e0664 service swift] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:37:51.426738 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f3d56-6cef-4db6-817a-48bedd4e0664 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:37:51.436567 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f3d56-6cef-4db6-817a-48bedd4e0664 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:37:51.438158 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f3d56-6cef-4db6-817a-48bedd4e0664 service swift] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:37:51.446675 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1315/2602] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Jul 18 06:37:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 06:38:01.229945 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=56 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:38:01.443215 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=80 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:38:19.714991 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81711b2d-4372-4911-b474-aa28f47babc9 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:38:19.715522 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81711b2d-4372-4911-b474-aa28f47babc9 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:38:19.715620 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81711b2d-4372-4911-b474-aa28f47babc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:38:19.736574 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-81711b2d-4372-4911-b474-aa28f47babc9 None None] MFA Rules not processed for user `b84dde57ff8e47419866377a819bea21`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:38:19.736574 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-81711b2d-4372-4911-b474-aa28f47babc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:38:19.742998 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1288/2603] 10.222.0.40 () {58 vars in 967 bytes} [Mon Jul 18 06:38:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:38:19.756480 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f3d56-6cef-4db6-817a-48bedd4e0664 service swift] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:38:19.767822 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f3d56-6cef-4db6-817a-48bedd4e0664 service swift] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:38:19.773372 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f6e8e3-b2b7-43d1-a52f-f2405368e657 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0681f5fbea6d415c9ad2f3afae256a5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:38:19.774695 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6e8e3-b2b7-43d1-a52f-f2405368e657 service swift] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:38:19.775597 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6e8e3-b2b7-43d1-a52f-f2405368e657 service swift] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:38:19.775882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6e8e3-b2b7-43d1-a52f-f2405368e657 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:38:19.785495 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f6e8e3-b2b7-43d1-a52f-f2405368e657 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:38:19.787082 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f6e8e3-b2b7-43d1-a52f-f2405368e657 service swift] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:38:19.796698 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1316/2604] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Jul 18 06:38:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 06:38:29.729223 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:38:29.792967 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=6 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:39:04.194827 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81711b2d-4372-4911-b474-aa28f47babc9 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:39:04.194827 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-81711b2d-4372-4911-b474-aa28f47babc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:39:04.217704 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81711b2d-4372-4911-b474-aa28f47babc9 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:39:04.224789 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024f6fa-ba93-4b4c-9101-9fde64edd6d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:39:04.225488 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024f6fa-ba93-4b4c-9101-9fde64edd6d9 service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:39:04.225742 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024f6fa-ba93-4b4c-9101-9fde64edd6d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:39:04.226025 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024f6fa-ba93-4b4c-9101-9fde64edd6d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:39:04.228549 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4024f6fa-ba93-4b4c-9101-9fde64edd6d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:39:04.244599 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4024f6fa-ba93-4b4c-9101-9fde64edd6d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:39:04.249741 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4024f6fa-ba93-4b4c-9101-9fde64edd6d9 service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:39:04.260438 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1289/2605] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:39:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:39:14.254575 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=6 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:39:39.980340 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f6e8e3-b2b7-43d1-a52f-f2405368e657 service swift] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:39:39.981791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-b7f6e8e3-b2b7-43d1-a52f-f2405368e657 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:39:39.995318 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f6e8e3-b2b7-43d1-a52f-f2405368e657 service swift] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:39:39.998986 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270b5613-6c36-40e9-bc12-7cb771a26386 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:39:40.000441 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b5613-6c36-40e9-bc12-7cb771a26386 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:39:40.000934 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b5613-6c36-40e9-bc12-7cb771a26386 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:39:40.001224 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b5613-6c36-40e9-bc12-7cb771a26386 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:39:40.002985 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-270b5613-6c36-40e9-bc12-7cb771a26386 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:39:40.084772 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270b5613-6c36-40e9-bc12-7cb771a26386 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:39:40.086208 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270b5613-6c36-40e9-bc12-7cb771a26386 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:39:40.104565 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1317/2606] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:39:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:39:50.101896 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=41 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:40:16.331629 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024f6fa-ba93-4b4c-9101-9fde64edd6d9 service nova] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:40:16.342721 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024f6fa-ba93-4b4c-9101-9fde64edd6d9 service nova] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:40:16.347895 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587e873a-d0e2-4f5c-b9f9-7bfde63dad25 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3560534427544cdabfd0ba702a0a165', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:40:16.348558 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e873a-d0e2-4f5c-b9f9-7bfde63dad25 service placement] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:40:16.348874 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e873a-d0e2-4f5c-b9f9-7bfde63dad25 service placement] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:40:16.349146 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e873a-d0e2-4f5c-b9f9-7bfde63dad25 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:40:16.350903 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-587e873a-d0e2-4f5c-b9f9-7bfde63dad25 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:40:16.363211 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587e873a-d0e2-4f5c-b9f9-7bfde63dad25 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:40:16.364820 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587e873a-d0e2-4f5c-b9f9-7bfde63dad25 service placement] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:40:16.374894 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1290/2607] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:40:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:40:26.370605 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=6 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:40:52.827243 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270b5613-6c36-40e9-bc12-7cb771a26386 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:40:52.839994 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270b5613-6c36-40e9-bc12-7cb771a26386 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:40:52.845580 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded685a6-ea9e-4d15-8b8c-3e24d985a625 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:40:52.846222 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded685a6-ea9e-4d15-8b8c-3e24d985a625 service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:40:52.846513 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded685a6-ea9e-4d15-8b8c-3e24d985a625 service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:40:52.847312 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded685a6-ea9e-4d15-8b8c-3e24d985a625 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:40:52.855582 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded685a6-ea9e-4d15-8b8c-3e24d985a625 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:40:52.857233 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded685a6-ea9e-4d15-8b8c-3e24d985a625 service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:40:52.885124 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1318/2608] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:40:52 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:41:02.882444 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=13 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:41:07.552281 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587e873a-d0e2-4f5c-b9f9-7bfde63dad25 service placement] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:41:07.562013 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587e873a-d0e2-4f5c-b9f9-7bfde63dad25 service placement] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:41:07.565712 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be7ede-2e7b-4544-b836-badde8d7f492 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:41:07.566346 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be7ede-2e7b-4544-b836-badde8d7f492 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:41:07.566660 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be7ede-2e7b-4544-b836-badde8d7f492 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:41:07.566941 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be7ede-2e7b-4544-b836-badde8d7f492 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:41:07.568736 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-52be7ede-2e7b-4544-b836-badde8d7f492 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:41:07.627055 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52be7ede-2e7b-4544-b836-badde8d7f492 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:41:07.628604 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52be7ede-2e7b-4544-b836-badde8d7f492 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:41:07.637545 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1291/2609] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:41:07 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:41:10.474914 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded685a6-ea9e-4d15-8b8c-3e24d985a625 service nova] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:41:10.485948 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded685a6-ea9e-4d15-8b8c-3e24d985a625 service nova] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:41:10.491214 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3525fe-c5b4-4b43-8d09-9e3851b016e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:41:10.492098 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3525fe-c5b4-4b43-8d09-9e3851b016e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:41:10.493118 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3525fe-c5b4-4b43-8d09-9e3851b016e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:41:10.493587 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3525fe-c5b4-4b43-8d09-9e3851b016e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:41:10.506959 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3525fe-c5b4-4b43-8d09-9e3851b016e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:41:10.508398 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3525fe-c5b4-4b43-8d09-9e3851b016e5 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:41:10.520369 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1319/2610] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:41:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:41:11.686326 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be7ede-2e7b-4544-b836-badde8d7f492 service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:41:11.697452 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be7ede-2e7b-4544-b836-badde8d7f492 service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:41:11.701687 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc821fc2-f416-4518-aada-905e1d4874b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:41:11.701958 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc821fc2-f416-4518-aada-905e1d4874b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:41:11.702251 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc821fc2-f416-4518-aada-905e1d4874b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:41:11.702536 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc821fc2-f416-4518-aada-905e1d4874b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:41:11.718179 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc821fc2-f416-4518-aada-905e1d4874b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:41:11.718179 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc821fc2-f416-4518-aada-905e1d4874b4 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:41:11.731324 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1292/2611] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:41:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:41:20.514921 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=6 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:41:21.724751 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=34 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:43:07.831340 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3525fe-c5b4-4b43-8d09-9e3851b016e5 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:43:07.841698 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3525fe-c5b4-4b43-8d09-9e3851b016e5 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:43:07.845417 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8beb7-1c64-4f00-8fc5-9b508680dec3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:43:07.846259 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8beb7-1c64-4f00-8fc5-9b508680dec3 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:43:07.846556 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8beb7-1c64-4f00-8fc5-9b508680dec3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:43:07.846840 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8beb7-1c64-4f00-8fc5-9b508680dec3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:43:07.862788 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a8beb7-1c64-4f00-8fc5-9b508680dec3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:43:07.864270 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a8beb7-1c64-4f00-8fc5-9b508680dec3 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:43:07.884033 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1320/2612] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:43:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:43:16.992686 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc821fc2-f416-4518-aada-905e1d4874b4 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:43:17.001592 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc821fc2-f416-4518-aada-905e1d4874b4 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:43:17.005081 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee17c6b-5288-4f78-af54-26fda693762f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:43:17.005879 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee17c6b-5288-4f78-af54-26fda693762f service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:43:17.006176 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee17c6b-5288-4f78-af54-26fda693762f service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:43:17.006462 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee17c6b-5288-4f78-af54-26fda693762f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:43:17.014937 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee17c6b-5288-4f78-af54-26fda693762f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:43:17.016532 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee17c6b-5288-4f78-af54-26fda693762f service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:43:17.025175 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1293/2613] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:43:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:43:17.873694 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=6 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:43:27.022022 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=6 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:45:08.225148 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8beb7-1c64-4f00-8fc5-9b508680dec3 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:45:08.262731 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8beb7-1c64-4f00-8fc5-9b508680dec3 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:45:08.266357 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f0787b-613f-4a4f-b612-92924c68558c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:45:08.267004 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0787b-613f-4a4f-b612-92924c68558c service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:45:08.267288 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0787b-613f-4a4f-b612-92924c68558c service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:45:08.267564 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0787b-613f-4a4f-b612-92924c68558c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:45:08.288881 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f0787b-613f-4a4f-b612-92924c68558c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:45:08.290295 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f0787b-613f-4a4f-b612-92924c68558c service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:45:08.300157 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1321/2614] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:45:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:45:18.295738 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=24 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:45:24.265855 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee17c6b-5288-4f78-af54-26fda693762f service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:45:24.304983 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee17c6b-5288-4f78-af54-26fda693762f service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:45:24.310311 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c029df85-bc0d-422d-97de-ce44a501baf4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3560534427544cdabfd0ba702a0a165', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:45:24.310961 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c029df85-bc0d-422d-97de-ce44a501baf4 service placement] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:45:24.311279 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c029df85-bc0d-422d-97de-ce44a501baf4 service placement] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:45:24.311567 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c029df85-bc0d-422d-97de-ce44a501baf4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:45:24.320675 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c029df85-bc0d-422d-97de-ce44a501baf4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:45:24.327873 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c029df85-bc0d-422d-97de-ce44a501baf4 service placement] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:45:24.333465 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1294/2615] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:45:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:45:34.329046 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=18 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:47:08.414552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f0787b-613f-4a4f-b612-92924c68558c service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:47:08.416105 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-50f0787b-613f-4a4f-b612-92924c68558c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:47:08.493242 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f0787b-613f-4a4f-b612-92924c68558c service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:47:08.497730 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b8200c-3b90-4e68-8783-470c304d4d4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:47:08.498384 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8200c-3b90-4e68-8783-470c304d4d4f service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:47:08.498672 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8200c-3b90-4e68-8783-470c304d4d4f service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:47:08.498953 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8200c-3b90-4e68-8783-470c304d4d4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:47:08.507480 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b8200c-3b90-4e68-8783-470c304d4d4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:47:08.508956 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b8200c-3b90-4e68-8783-470c304d4d4f service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:47:08.535927 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1322/2616] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:47:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:47:08.585288 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c029df85-bc0d-422d-97de-ce44a501baf4 service placement] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:47:08.594627 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c029df85-bc0d-422d-97de-ce44a501baf4 service placement] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:47:08.598160 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a887417-a536-410c-a5d6-583893ed2790 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:47:08.598791 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a887417-a536-410c-a5d6-583893ed2790 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:47:08.599075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a887417-a536-410c-a5d6-583893ed2790 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:47:08.599355 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a887417-a536-410c-a5d6-583893ed2790 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:47:08.601242 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7a887417-a536-410c-a5d6-583893ed2790 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:47:08.690406 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a887417-a536-410c-a5d6-583893ed2790 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:47:08.691905 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a887417-a536-410c-a5d6-583893ed2790 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:47:08.712798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1295/2617] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:47:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:47:08.759390 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b8200c-3b90-4e68-8783-470c304d4d4f service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:47:08.764397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a887417-a536-410c-a5d6-583893ed2790 service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:47:08.766231 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b8200c-3b90-4e68-8783-470c304d4d4f service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:47:08.769319 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d875b2e5-366f-4884-8049-037b525fa2e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:47:08.769997 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875b2e5-366f-4884-8049-037b525fa2e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:47:08.770288 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875b2e5-366f-4884-8049-037b525fa2e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:47:08.770572 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875b2e5-366f-4884-8049-037b525fa2e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:47:08.772514 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d875b2e5-366f-4884-8049-037b525fa2e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:47:08.773951 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a887417-a536-410c-a5d6-583893ed2790 service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:47:08.779531 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68225a5-9b92-48d1-9cc0-6421d72ea51d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:47:08.780171 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68225a5-9b92-48d1-9cc0-6421d72ea51d service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:47:08.780486 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68225a5-9b92-48d1-9cc0-6421d72ea51d service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:47:08.780779 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68225a5-9b92-48d1-9cc0-6421d72ea51d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:47:08.782634 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f68225a5-9b92-48d1-9cc0-6421d72ea51d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:47:08.786794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d875b2e5-366f-4884-8049-037b525fa2e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:47:08.788235 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d875b2e5-366f-4884-8049-037b525fa2e4 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:47:08.799594 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1323/2618] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:47:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:47:08.856727 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68225a5-9b92-48d1-9cc0-6421d72ea51d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:47:08.858215 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68225a5-9b92-48d1-9cc0-6421d72ea51d service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:47:08.867922 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1296/2619] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:47:08 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 06:47:18.794497 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=48 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:47:18.864010 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=79 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:48:06.853048 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d875b2e5-366f-4884-8049-037b525fa2e4 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:48:06.864645 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d875b2e5-366f-4884-8049-037b525fa2e4 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:48:06.870254 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a5dc1-6be8-471c-86fd-e1f051a3b6c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:48:06.870922 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a5dc1-6be8-471c-86fd-e1f051a3b6c7 service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:48:06.871234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a5dc1-6be8-471c-86fd-e1f051a3b6c7 service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:48:06.871514 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a5dc1-6be8-471c-86fd-e1f051a3b6c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:48:06.873417 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-7d1a5dc1-6be8-471c-86fd-e1f051a3b6c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:48:06.951601 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1a5dc1-6be8-471c-86fd-e1f051a3b6c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:48:06.953226 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1a5dc1-6be8-471c-86fd-e1f051a3b6c7 service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:48:06.972166 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1324/2620] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:48:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:48:16.969464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=41 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:48:19.968790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68225a5-9b92-48d1-9cc0-6421d72ea51d service nova] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:48:19.970385 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f68225a5-9b92-48d1-9cc0-6421d72ea51d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:48:20.048977 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68225a5-9b92-48d1-9cc0-6421d72ea51d service nova] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:48:20.052635 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e404e1fa-53f8-4b73-9b2e-97c205346b85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:48:20.053283 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404e1fa-53f8-4b73-9b2e-97c205346b85 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:48:20.053577 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404e1fa-53f8-4b73-9b2e-97c205346b85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:48:20.053860 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404e1fa-53f8-4b73-9b2e-97c205346b85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:48:20.055885 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e404e1fa-53f8-4b73-9b2e-97c205346b85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:48:20.065264 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e404e1fa-53f8-4b73-9b2e-97c205346b85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:48:20.066675 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e404e1fa-53f8-4b73-9b2e-97c205346b85 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:48:20.074704 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1297/2621] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:48:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:48:30.071173 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=35 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:50:08.806934 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a5dc1-6be8-471c-86fd-e1f051a3b6c7 service nova] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:50:08.808550 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-7d1a5dc1-6be8-471c-86fd-e1f051a3b6c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:50:08.820586 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a5dc1-6be8-471c-86fd-e1f051a3b6c7 service nova] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:50:08.824121 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3646e1e-195d-4307-bd8a-948557b918d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:50:08.824798 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3646e1e-195d-4307-bd8a-948557b918d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:50:08.825093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3646e1e-195d-4307-bd8a-948557b918d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:50:08.825379 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3646e1e-195d-4307-bd8a-948557b918d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:50:08.827182 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d3646e1e-195d-4307-bd8a-948557b918d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:50:08.906628 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3646e1e-195d-4307-bd8a-948557b918d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:50:08.908064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3646e1e-195d-4307-bd8a-948557b918d7 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:50:08.926952 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1325/2622] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:50:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:50:18.923958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=41 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:50:31.242986 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e404e1fa-53f8-4b73-9b2e-97c205346b85 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:50:31.244589 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e404e1fa-53f8-4b73-9b2e-97c205346b85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:50:31.336303 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e404e1fa-53f8-4b73-9b2e-97c205346b85 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:50:31.341612 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed660a-b88f-4ad4-8554-00b1f61b64ce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3560534427544cdabfd0ba702a0a165', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:50:31.342268 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed660a-b88f-4ad4-8554-00b1f61b64ce service placement] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:50:31.342556 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed660a-b88f-4ad4-8554-00b1f61b64ce service placement] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:50:31.342836 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed660a-b88f-4ad4-8554-00b1f61b64ce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:50:31.344760 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-72ed660a-b88f-4ad4-8554-00b1f61b64ce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:50:31.356729 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ed660a-b88f-4ad4-8554-00b1f61b64ce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:50:31.358389 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ed660a-b88f-4ad4-8554-00b1f61b64ce service placement] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:50:31.368537 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1298/2623] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:50:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:50:41.364733 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=37 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:52:08.179222 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3646e1e-195d-4307-bd8a-948557b918d7 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:52:08.187972 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3646e1e-195d-4307-bd8a-948557b918d7 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:52:08.191702 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48a128-eddb-42f2-8698-b9486bdea931 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:52:08.192336 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48a128-eddb-42f2-8698-b9486bdea931 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:52:08.192652 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48a128-eddb-42f2-8698-b9486bdea931 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:52:08.192932 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48a128-eddb-42f2-8698-b9486bdea931 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:52:08.201240 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f48a128-eddb-42f2-8698-b9486bdea931 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:52:08.202696 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f48a128-eddb-42f2-8698-b9486bdea931 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:52:08.229845 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1326/2624] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:52:08 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:52:08.550128 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed660a-b88f-4ad4-8554-00b1f61b64ce service placement] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:52:08.559422 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed660a-b88f-4ad4-8554-00b1f61b64ce service placement] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:52:08.563290 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721d0c1-7db4-47d9-a95d-55b1c4a69422 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:52:08.563967 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721d0c1-7db4-47d9-a95d-55b1c4a69422 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:52:08.564254 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721d0c1-7db4-47d9-a95d-55b1c4a69422 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:52:08.564578 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721d0c1-7db4-47d9-a95d-55b1c4a69422 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:52:08.566661 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-5721d0c1-7db4-47d9-a95d-55b1c4a69422 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:52:08.637340 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5721d0c1-7db4-47d9-a95d-55b1c4a69422 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:52:08.638781 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5721d0c1-7db4-47d9-a95d-55b1c4a69422 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:52:08.648499 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1299/2625] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:52:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:52:18.227259 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=13 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:52:18.644299 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=33 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:53:07.657994 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48a128-eddb-42f2-8698-b9486bdea931 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:53:07.659613 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4f48a128-eddb-42f2-8698-b9486bdea931 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:53:07.676038 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48a128-eddb-42f2-8698-b9486bdea931 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:53:07.681814 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6340a-f67f-44df-92f0-bc2ff2f56966 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:53:07.682467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6340a-f67f-44df-92f0-bc2ff2f56966 service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:53:07.682755 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6340a-f67f-44df-92f0-bc2ff2f56966 service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:53:07.683038 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6340a-f67f-44df-92f0-bc2ff2f56966 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:53:07.692204 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff6340a-f67f-44df-92f0-bc2ff2f56966 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:53:07.693843 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff6340a-f67f-44df-92f0-bc2ff2f56966 service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:53:07.702856 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1327/2626] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:53:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:53:17.698616 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=6 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:53:26.963382 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721d0c1-7db4-47d9-a95d-55b1c4a69422 service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:53:26.973146 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721d0c1-7db4-47d9-a95d-55b1c4a69422 service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:53:26.976786 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66500bda-2710-4f17-92cd-98747284de14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:53:26.977430 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66500bda-2710-4f17-92cd-98747284de14 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:53:26.977716 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66500bda-2710-4f17-92cd-98747284de14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:53:26.977999 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66500bda-2710-4f17-92cd-98747284de14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:53:26.985297 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66500bda-2710-4f17-92cd-98747284de14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:53:26.986748 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66500bda-2710-4f17-92cd-98747284de14 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:53:26.994712 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1300/2627] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:53:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:53:36.991195 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=6 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:53:41.647275 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6340a-f67f-44df-92f0-bc2ff2f56966 service nova] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:53:41.656858 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6340a-f67f-44df-92f0-bc2ff2f56966 service nova] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:53:41.661255 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92810306-0876-4f55-a898-726d9a4eb054 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:53:41.661890 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92810306-0876-4f55-a898-726d9a4eb054 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:53:41.662180 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92810306-0876-4f55-a898-726d9a4eb054 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:53:41.662451 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92810306-0876-4f55-a898-726d9a4eb054 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:53:41.664196 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-92810306-0876-4f55-a898-726d9a4eb054 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:53:41.750712 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92810306-0876-4f55-a898-726d9a4eb054 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:53:41.752824 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92810306-0876-4f55-a898-726d9a4eb054 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:53:41.772851 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1328/2628] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:53:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:53:41.954496 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66500bda-2710-4f17-92cd-98747284de14 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:53:41.964375 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66500bda-2710-4f17-92cd-98747284de14 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:53:41.968540 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8cccf-1a30-4222-87aa-de4155abda10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:53:41.969289 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8cccf-1a30-4222-87aa-de4155abda10 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:53:41.969646 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8cccf-1a30-4222-87aa-de4155abda10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:53:41.969996 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8cccf-1a30-4222-87aa-de4155abda10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:53:41.981174 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc8cccf-1a30-4222-87aa-de4155abda10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:53:41.983418 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc8cccf-1a30-4222-87aa-de4155abda10 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:53:41.994119 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1301/2629] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:53:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:53:43.513318 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92810306-0876-4f55-a898-726d9a4eb054 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:53:43.522734 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92810306-0876-4f55-a898-726d9a4eb054 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:53:43.526363 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3f759-0fd6-4dd3-af7e-c390734f31b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:53:43.527010 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3f759-0fd6-4dd3-af7e-c390734f31b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:53:43.527292 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3f759-0fd6-4dd3-af7e-c390734f31b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:53:43.527568 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3f759-0fd6-4dd3-af7e-c390734f31b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:53:43.541952 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f3f759-0fd6-4dd3-af7e-c390734f31b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:53:43.544375 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f3f759-0fd6-4dd3-af7e-c390734f31b3 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:53:43.556991 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1329/2630] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:53:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:53:45.548802 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8cccf-1a30-4222-87aa-de4155abda10 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:53:45.560313 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8cccf-1a30-4222-87aa-de4155abda10 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:53:45.566112 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b884d-0fc7-4711-94ec-bc5ae7d2a308 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3560534427544cdabfd0ba702a0a165', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:53:45.566755 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b884d-0fc7-4711-94ec-bc5ae7d2a308 service placement] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:53:45.567048 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b884d-0fc7-4711-94ec-bc5ae7d2a308 service placement] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:53:45.567331 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b884d-0fc7-4711-94ec-bc5ae7d2a308 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:53:45.569164 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-818b884d-0fc7-4711-94ec-bc5ae7d2a308 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:53:45.581544 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818b884d-0fc7-4711-94ec-bc5ae7d2a308 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:53:45.583237 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818b884d-0fc7-4711-94ec-bc5ae7d2a308 service placement] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:53:45.594059 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1302/2631] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:53:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:53:45.687897 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3f759-0fd6-4dd3-af7e-c390734f31b3 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:53:45.689734 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d0f3f759-0fd6-4dd3-af7e-c390734f31b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:53:45.704537 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3f759-0fd6-4dd3-af7e-c390734f31b3 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:53:45.710225 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2098bc66-efe3-4fd8-812d-cb927f80952a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:53:45.711258 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098bc66-efe3-4fd8-812d-cb927f80952a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:53:45.711598 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098bc66-efe3-4fd8-812d-cb927f80952a admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:53:45.711878 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098bc66-efe3-4fd8-812d-cb927f80952a admin admin] PATH_INFO: `/v3/users/936ea0959e924aac95cf474687e35c78` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:53:45.713122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2098bc66-efe3-4fd8-812d-cb927f80952a admin admin] RBAC: Authorizing `identity:delete_user(user_id=936ea0959e924aac95cf474687e35c78)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:53:45.715160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2098bc66-efe3-4fd8-812d-cb927f80952a admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:53:45.894656 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-2098bc66-efe3-4fd8-812d-cb927f80952a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '936ea0959e924aac95cf474687e35c78'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:53:45.901623 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-2098bc66-efe3-4fd8-812d-cb927f80952a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '936ea0959e924aac95cf474687e35c78'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:53:45.902284 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-2098bc66-efe3-4fd8-812d-cb927f80952a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '936ea0959e924aac95cf474687e35c78'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:53:45.902892 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-2098bc66-efe3-4fd8-812d-cb927f80952a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '936ea0959e924aac95cf474687e35c78'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:53:45.914895 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-2098bc66-efe3-4fd8-812d-cb927f80952a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '936ea0959e924aac95cf474687e35c78'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:53:45.996816 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1330/2632] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 06:53:45 2022] DELETE /identity/v3/users/936ea0959e924aac95cf474687e35c78 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:53:46.312787 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b884d-0fc7-4711-94ec-bc5ae7d2a308 service placement] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:53:46.324138 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b884d-0fc7-4711-94ec-bc5ae7d2a308 service placement] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:53:46.387534 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdb15bd-d5d9-4fa5-94a9-c96398e12ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:53:46.388389 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb15bd-d5d9-4fa5-94a9-c96398e12ec0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:53:46.388706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb15bd-d5d9-4fa5-94a9-c96398e12ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:53:46.389004 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb15bd-d5d9-4fa5-94a9-c96398e12ec0 admin admin] PATH_INFO: `/v3/projects/31c9d72882974d3f96b4e5d3b4ec701d` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:53:46.390184 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdb15bd-d5d9-4fa5-94a9-c96398e12ec0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31c9d72882974d3f96b4e5d3b4ec701d)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:53:46.392084 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdb15bd-d5d9-4fa5-94a9-c96398e12ec0 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:53:46.515342 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1fdb15bd-d5d9-4fa5-94a9-c96398e12ec0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31c9d72882974d3f96b4e5d3b4ec701d', 'request_id': 'req-1fdb15bd-d5d9-4fa5-94a9-c96398e12ec0'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:53:46.521469 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1fdb15bd-d5d9-4fa5-94a9-c96398e12ec0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31c9d72882974d3f96b4e5d3b4ec701d', 'request_id': 'req-1fdb15bd-d5d9-4fa5-94a9-c96398e12ec0'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:53:46.590816 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1fdb15bd-d5d9-4fa5-94a9-c96398e12ec0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31c9d72882974d3f96b4e5d3b4ec701d', 'request_id': 'req-1fdb15bd-d5d9-4fa5-94a9-c96398e12ec0'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:53:46.602829 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1fdb15bd-d5d9-4fa5-94a9-c96398e12ec0 admin admin] The token cache is being invalidate because project 31c9d72882974d3f96b4e5d3b4ec701d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 06:53:46.603123 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1fdb15bd-d5d9-4fa5-94a9-c96398e12ec0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:53:46.603765 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1fdb15bd-d5d9-4fa5-94a9-c96398e12ec0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 06:53:46.605332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1303/2633] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 06:53:46 2022] DELETE /identity/v3/projects/31c9d72882974d3f96b4e5d3b4ec701d => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:53:46.611556 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5f7b7-eb17-4722-ba93-5bcbc32d1a2a None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:53:46.611853 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5f7b7-eb17-4722-ba93-5bcbc32d1a2a None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:53:46.612139 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5f7b7-eb17-4722-ba93-5bcbc32d1a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:53:46.644835 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:53:46.672192 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-d8a5f7b7-eb17-4722-ba93-5bcbc32d1a2a None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:53:46.673481 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d8a5f7b7-eb17-4722-ba93-5bcbc32d1a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:53:46.735813 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1331/2634] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:53:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:53:46.744504 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d202d-f620-451e-a3ef-48751160c245 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:53:46.744504 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d202d-f620-451e-a3ef-48751160c245 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:53:46.744504 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d202d-f620-451e-a3ef-48751160c245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:53:46.762668 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-0d5d202d-f620-451e-a3ef-48751160c245 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:53:46.763882 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-0d5d202d-f620-451e-a3ef-48751160c245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:53:46.773343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1304/2635] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:53:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 06:53:46.777315 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a5f7b7-eb17-4722-ba93-5bcbc32d1a2a None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:53:46.788000 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a5f7b7-eb17-4722-ba93-5bcbc32d1a2a None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:53:46.792909 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9ebf9-83f8-4ee6-9274-4896ead8cdce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:53:46.793581 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9ebf9-83f8-4ee6-9274-4896ead8cdce admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:53:46.793869 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9ebf9-83f8-4ee6-9274-4896ead8cdce admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:53:46.794151 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9ebf9-83f8-4ee6-9274-4896ead8cdce admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:53:46.795134 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e9ebf9-83f8-4ee6-9274-4896ead8cdce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 06:53:46.795454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e9ebf9-83f8-4ee6-9274-4896ead8cdce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:53:46.797108 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e9ebf9-83f8-4ee6-9274-4896ead8cdce admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:53:46.812946 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1332/2636] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 06:53:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 06:53:46.814071 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d202d-f620-451e-a3ef-48751160c245 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:53:46.823692 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d202d-f620-451e-a3ef-48751160c245 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:53:46.828286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de40e3-690d-45ed-b0b0-6723b2f2b795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:53:46.828989 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de40e3-690d-45ed-b0b0-6723b2f2b795 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:53:46.829272 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de40e3-690d-45ed-b0b0-6723b2f2b795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:53:46.829547 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de40e3-690d-45ed-b0b0-6723b2f2b795 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:53:46.831067 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9de40e3-690d-45ed-b0b0-6723b2f2b795 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:53:46.832976 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9de40e3-690d-45ed-b0b0-6723b2f2b795 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:53:46.940445 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1305/2637] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 06:53:46 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:53:46.944668 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9ebf9-83f8-4ee6-9274-4896ead8cdce admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:53:46.955648 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9ebf9-83f8-4ee6-9274-4896ead8cdce admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:53:47.010312 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c185c-ea22-4a65-8267-6642478b0f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:53:47.010987 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c185c-ea22-4a65-8267-6642478b0f56 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:53:47.011277 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c185c-ea22-4a65-8267-6642478b0f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:53:47.011572 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c185c-ea22-4a65-8267-6642478b0f56 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:53:47.012261 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c185c-ea22-4a65-8267-6642478b0f56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:53:47.014255 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c185c-ea22-4a65-8267-6642478b0f56 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:53:47.103263 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1333/2638] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 06:53:46 2022] POST /identity/v3/users => generated 293 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 06:53:47.107243 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de40e3-690d-45ed-b0b0-6723b2f2b795 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:53:47.119112 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de40e3-690d-45ed-b0b0-6723b2f2b795 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:53:47.123987 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b747cc6-98d3-4f5a-9143-4fa4ff02dbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:53:47.124677 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b747cc6-98d3-4f5a-9143-4fa4ff02dbe3 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:53:47.124969 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b747cc6-98d3-4f5a-9143-4fa4ff02dbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:53:47.125247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b747cc6-98d3-4f5a-9143-4fa4ff02dbe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:53:47.125974 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b747cc6-98d3-4f5a-9143-4fa4ff02dbe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:53:47.128324 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b747cc6-98d3-4f5a-9143-4fa4ff02dbe3 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:53:47.138721 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1306/2639] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:53:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:53:47.143359 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c185c-ea22-4a65-8267-6642478b0f56 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:53:47.153417 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c185c-ea22-4a65-8267-6642478b0f56 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:53:47.158084 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d71705a-5677-4f6f-8282-75ba6f1c8ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:53:47.158771 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d71705a-5677-4f6f-8282-75ba6f1c8ce3 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:53:47.159064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d71705a-5677-4f6f-8282-75ba6f1c8ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:53:47.159349 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d71705a-5677-4f6f-8282-75ba6f1c8ce3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:53:47.160080 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d71705a-5677-4f6f-8282-75ba6f1c8ce3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:53:47.161717 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d71705a-5677-4f6f-8282-75ba6f1c8ce3 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:53:47.171586 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1334/2640] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 06:53:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 06:53:47.175499 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b747cc6-98d3-4f5a-9143-4fa4ff02dbe3 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:53:47.185329 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b747cc6-98d3-4f5a-9143-4fa4ff02dbe3 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:53:47.190574 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa0ca76-7e31-456f-bafb-2ca7a855573e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:53:47.191482 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0ca76-7e31-456f-bafb-2ca7a855573e admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:53:47.191767 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0ca76-7e31-456f-bafb-2ca7a855573e admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:53:47.192047 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0ca76-7e31-456f-bafb-2ca7a855573e admin admin] PATH_INFO: `/v3/projects/4f560c50833d4fb29fa6e0317eb96a47/users/e3ee9d190f9e45e2bc546d8b24f6568c/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:53:47.204006 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:53:47.207724 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa0ca76-7e31-456f-bafb-2ca7a855573e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f560c50833d4fb29fa6e0317eb96a47, user_id=e3ee9d190f9e45e2bc546d8b24f6568c, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:53:47.210414 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa0ca76-7e31-456f-bafb-2ca7a855573e admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:53:47.292587 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1307/2641] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 06:53:47 2022] PUT /identity/v3/projects/4f560c50833d4fb29fa6e0317eb96a47/users/e3ee9d190f9e45e2bc546d8b24f6568c/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 06:53:47.303880 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639651f-910e-432a-af89-6a83878383ef None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:53:47.304171 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639651f-910e-432a-af89-6a83878383ef None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:53:47.304476 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639651f-910e-432a-af89-6a83878383ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:53:47.332776 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-c639651f-910e-432a-af89-6a83878383ef None None] MFA Rules not processed for user `e3ee9d190f9e45e2bc546d8b24f6568c`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 06:53:47.334111 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c639651f-910e-432a-af89-6a83878383ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:53:47.387695 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:53:47.405216 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1335/2642] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 06:53:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 06:53:47.416474 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa0ca76-7e31-456f-bafb-2ca7a855573e admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:53:47.419529 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-2fa0ca76-7e31-456f-bafb-2ca7a855573e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:53:47.491713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa0ca76-7e31-456f-bafb-2ca7a855573e admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:53:47.497390 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd11a4ac-7137-4f19-8062-c7dfa146b13a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:53:47.498027 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11a4ac-7137-4f19-8062-c7dfa146b13a service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:53:47.498310 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11a4ac-7137-4f19-8062-c7dfa146b13a service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:53:47.498588 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11a4ac-7137-4f19-8062-c7dfa146b13a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:53:47.507143 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd11a4ac-7137-4f19-8062-c7dfa146b13a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:53:47.508733 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd11a4ac-7137-4f19-8062-c7dfa146b13a service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:53:47.517482 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1308/2643] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:53:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:53:57.513478 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=37,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:53:57.520471 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=5 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:55:35.255382 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c639651f-910e-432a-af89-6a83878383ef None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:55:35.257040 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c639651f-910e-432a-af89-6a83878383ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:55:35.382829 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c639651f-910e-432a-af89-6a83878383ef None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:55:35.387725 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a090c-3039-4eb5-9d15-552c409f3cea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3560534427544cdabfd0ba702a0a165', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:55:35.388364 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a090c-3039-4eb5-9d15-552c409f3cea service placement] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:55:35.388681 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a090c-3039-4eb5-9d15-552c409f3cea service placement] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:55:35.388955 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a090c-3039-4eb5-9d15-552c409f3cea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:55:35.391424 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-b30a090c-3039-4eb5-9d15-552c409f3cea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:55:35.402548 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30a090c-3039-4eb5-9d15-552c409f3cea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:55:35.404194 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30a090c-3039-4eb5-9d15-552c409f3cea service placement] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:55:35.414339 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1336/2644] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:55:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:55:45.410113 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=54 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:57:08.413765 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd11a4ac-7137-4f19-8062-c7dfa146b13a service nova] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:57:08.455047 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd11a4ac-7137-4f19-8062-c7dfa146b13a service nova] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:57:08.460890 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118eaf67-fde5-4291-a567-c3e5ce8c7964 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:57:08.461530 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118eaf67-fde5-4291-a567-c3e5ce8c7964 service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:57:08.461822 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118eaf67-fde5-4291-a567-c3e5ce8c7964 service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:57:08.462134 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118eaf67-fde5-4291-a567-c3e5ce8c7964 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:57:08.463911 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-118eaf67-fde5-4291-a567-c3e5ce8c7964 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:57:08.530612 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118eaf67-fde5-4291-a567-c3e5ce8c7964 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:57:08.532193 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118eaf67-fde5-4291-a567-c3e5ce8c7964 service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:57:08.561256 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1309/2645] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:57:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:57:08.622998 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a090c-3039-4eb5-9d15-552c409f3cea service placement] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:57:08.625481 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-b30a090c-3039-4eb5-9d15-552c409f3cea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:57:08.721220 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a090c-3039-4eb5-9d15-552c409f3cea service placement] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:57:08.724924 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5760e8cc-03a8-4a57-8eb0-d631c4385728 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:57:08.725569 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760e8cc-03a8-4a57-8eb0-d631c4385728 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:57:08.725851 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760e8cc-03a8-4a57-8eb0-d631c4385728 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:57:08.726124 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760e8cc-03a8-4a57-8eb0-d631c4385728 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:57:08.727859 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-5760e8cc-03a8-4a57-8eb0-d631c4385728 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:57:08.785215 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5760e8cc-03a8-4a57-8eb0-d631c4385728 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:57:08.786642 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5760e8cc-03a8-4a57-8eb0-d631c4385728 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:57:08.796197 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1337/2646] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:57:08 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:57:18.557902 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=53 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:57:18.791420 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=61 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:57:44.246703 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118eaf67-fde5-4291-a567-c3e5ce8c7964 service nova] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:57:44.248644 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-118eaf67-fde5-4291-a567-c3e5ce8c7964 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:57:44.290626 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118eaf67-fde5-4291-a567-c3e5ce8c7964 service nova] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:57:44.296926 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e645c-8d16-4b2e-9c03-9e8aa97dfe2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:57:44.297572 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e645c-8d16-4b2e-9c03-9e8aa97dfe2e service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:57:44.297859 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e645c-8d16-4b2e-9c03-9e8aa97dfe2e service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:57:44.298138 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e645c-8d16-4b2e-9c03-9e8aa97dfe2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:57:44.299913 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a92e645c-8d16-4b2e-9c03-9e8aa97dfe2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:57:44.312047 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92e645c-8d16-4b2e-9c03-9e8aa97dfe2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:57:44.313748 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92e645c-8d16-4b2e-9c03-9e8aa97dfe2e service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:57:44.324758 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1310/2647] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:57:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:57:54.319871 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=18 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:58:29.931937 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5760e8cc-03a8-4a57-8eb0-d631c4385728 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:58:29.933367 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-5760e8cc-03a8-4a57-8eb0-d631c4385728 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:58:30.014066 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5760e8cc-03a8-4a57-8eb0-d631c4385728 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:58:30.017904 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718fc816-bfd1-4b84-95d2-d43b4e7d2e1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:58:30.018555 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718fc816-bfd1-4b84-95d2-d43b4e7d2e1e service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:58:30.018849 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718fc816-bfd1-4b84-95d2-d43b4e7d2e1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:58:30.019156 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718fc816-bfd1-4b84-95d2-d43b4e7d2e1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:58:30.020976 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-718fc816-bfd1-4b84-95d2-d43b4e7d2e1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:58:30.030650 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718fc816-bfd1-4b84-95d2-d43b4e7d2e1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:58:30.032079 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718fc816-bfd1-4b84-95d2-d43b4e7d2e1e service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:58:30.040121 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1338/2648] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:58:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:58:40.036455 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=35 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:58:41.955956 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e645c-8d16-4b2e-9c03-9e8aa97dfe2e service nova] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:58:41.965568 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e645c-8d16-4b2e-9c03-9e8aa97dfe2e service nova] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:58:41.969935 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073608b-6b81-4541-8e79-4c2a896ee937 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:58:41.970573 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073608b-6b81-4541-8e79-4c2a896ee937 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:58:41.970864 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073608b-6b81-4541-8e79-4c2a896ee937 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:58:41.971150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073608b-6b81-4541-8e79-4c2a896ee937 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:58:41.972960 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f073608b-6b81-4541-8e79-4c2a896ee937 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:58:41.987102 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f073608b-6b81-4541-8e79-4c2a896ee937 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:58:41.988525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f073608b-6b81-4541-8e79-4c2a896ee937 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:58:41.999960 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1311/2649] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:58:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:58:51.994668 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=6 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:59:43.057843 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718fc816-bfd1-4b84-95d2-d43b4e7d2e1e service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:59:43.067516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718fc816-bfd1-4b84-95d2-d43b4e7d2e1e service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:59:43.071081 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30295219-0e3c-454d-95f5-37fe9fd3eb5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:59:43.071718 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30295219-0e3c-454d-95f5-37fe9fd3eb5c service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:59:43.072009 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30295219-0e3c-454d-95f5-37fe9fd3eb5c service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:59:43.072290 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30295219-0e3c-454d-95f5-37fe9fd3eb5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:59:43.074134 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-30295219-0e3c-454d-95f5-37fe9fd3eb5c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:59:43.087787 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30295219-0e3c-454d-95f5-37fe9fd3eb5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:59:43.089244 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30295219-0e3c-454d-95f5-37fe9fd3eb5c service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:59:43.100931 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1339/2650] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:59:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:59:43.108473 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073608b-6b81-4541-8e79-4c2a896ee937 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:59:43.117475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073608b-6b81-4541-8e79-4c2a896ee937 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:59:43.120959 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58035e7e-eb8b-42dd-a84d-01d82ea743b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:59:43.121582 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58035e7e-eb8b-42dd-a84d-01d82ea743b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:59:43.121863 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58035e7e-eb8b-42dd-a84d-01d82ea743b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:59:43.122138 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58035e7e-eb8b-42dd-a84d-01d82ea743b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:59:43.130443 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58035e7e-eb8b-42dd-a84d-01d82ea743b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:59:43.131837 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58035e7e-eb8b-42dd-a84d-01d82ea743b5 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:59:43.140519 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1312/2651] 10.222.0.40 () {60 vars in 1396 bytes} [Mon Jul 18 06:59:43 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3252 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:59:43.201995 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30295219-0e3c-454d-95f5-37fe9fd3eb5c service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 06:59:43.210769 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30295219-0e3c-454d-95f5-37fe9fd3eb5c service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 06:59:43.214336 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58667237-5d4c-4584-998e-c5f981054db3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 06:59:43.214965 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58667237-5d4c-4584-998e-c5f981054db3 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 06:59:43.215250 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58667237-5d4c-4584-998e-c5f981054db3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 06:59:43.215527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58667237-5d4c-4584-998e-c5f981054db3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 06:59:43.218011 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-58667237-5d4c-4584-998e-c5f981054db3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 06:59:43.304080 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58667237-5d4c-4584-998e-c5f981054db3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 06:59:43.305473 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58667237-5d4c-4584-998e-c5f981054db3 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 06:59:43.324161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1340/2652] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 06:59:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 06:59:53.136530 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=6 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 06:59:53.321468 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=47 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:00:38.189745 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58035e7e-eb8b-42dd-a84d-01d82ea743b5 service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:00:38.191252 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-58035e7e-eb8b-42dd-a84d-01d82ea743b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:00:38.219965 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58035e7e-eb8b-42dd-a84d-01d82ea743b5 service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:00:38.224986 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b612a2a1-54c0-4303-b193-6e746da558f6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3560534427544cdabfd0ba702a0a165', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:00:38.225630 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b612a2a1-54c0-4303-b193-6e746da558f6 service placement] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:00:38.225920 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b612a2a1-54c0-4303-b193-6e746da558f6 service placement] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:00:38.226201 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b612a2a1-54c0-4303-b193-6e746da558f6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:00:38.234961 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b612a2a1-54c0-4303-b193-6e746da558f6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:00:38.236558 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b612a2a1-54c0-4303-b193-6e746da558f6 service placement] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:00:38.246409 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1313/2653] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:00:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:00:48.242177 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=11 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:02:08.794491 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58667237-5d4c-4584-998e-c5f981054db3 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:02:08.807537 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58667237-5d4c-4584-998e-c5f981054db3 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:02:08.813552 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9450b70c-d930-4f00-a472-2f865c3859ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:02:08.814195 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450b70c-d930-4f00-a472-2f865c3859ca service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:02:08.814479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450b70c-d930-4f00-a472-2f865c3859ca service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:02:08.814755 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450b70c-d930-4f00-a472-2f865c3859ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:02:08.822633 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9450b70c-d930-4f00-a472-2f865c3859ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:02:08.824210 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9450b70c-d930-4f00-a472-2f865c3859ca service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:02:08.851923 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1341/2654] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:02:08 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:02:18.849287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=13 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:03:33.933849 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b612a2a1-54c0-4303-b193-6e746da558f6 service placement] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:03:33.943590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b612a2a1-54c0-4303-b193-6e746da558f6 service placement] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:03:33.947452 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2791f1-c428-4e2f-bdd8-f2f56a987a02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:03:33.948144 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2791f1-c428-4e2f-bdd8-f2f56a987a02 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:03:33.948450 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2791f1-c428-4e2f-bdd8-f2f56a987a02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:03:33.948735 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2791f1-c428-4e2f-bdd8-f2f56a987a02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:03:33.956435 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2791f1-c428-4e2f-bdd8-f2f56a987a02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:03:33.957857 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2791f1-c428-4e2f-bdd8-f2f56a987a02 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:03:33.966573 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1314/2655] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:03:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:03:43.594429 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9450b70c-d930-4f00-a472-2f865c3859ca service nova] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:03:43.604680 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9450b70c-d930-4f00-a472-2f865c3859ca service nova] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:03:43.608161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d8b46-ab91-4508-9f61-e858c580aeba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:03:43.608824 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d8b46-ab91-4508-9f61-e858c580aeba service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:03:43.609108 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d8b46-ab91-4508-9f61-e858c580aeba service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:03:43.609383 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d8b46-ab91-4508-9f61-e858c580aeba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:03:43.617972 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d8b46-ab91-4508-9f61-e858c580aeba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:03:43.619385 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d8b46-ab91-4508-9f61-e858c580aeba service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:03:43.628067 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1342/2656] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:03:43 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:03:43.962928 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=6 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:03:45.156689 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2791f1-c428-4e2f-bdd8-f2f56a987a02 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:03:45.158172 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ed2791f1-c428-4e2f-bdd8-f2f56a987a02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:03:45.170611 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2791f1-c428-4e2f-bdd8-f2f56a987a02 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:03:45.174307 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb6ca94-32a1-4f27-8652-ac59f55a1a6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:03:45.174955 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6ca94-32a1-4f27-8652-ac59f55a1a6f service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:03:45.175242 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6ca94-32a1-4f27-8652-ac59f55a1a6f service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:03:45.175525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6ca94-32a1-4f27-8652-ac59f55a1a6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:03:45.187307 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb6ca94-32a1-4f27-8652-ac59f55a1a6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:03:45.188801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb6ca94-32a1-4f27-8652-ac59f55a1a6f service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:03:45.200895 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1315/2657] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:03:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:03:53.624109 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=6 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:03:55.195864 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=6 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:04:43.251484 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d8b46-ab91-4508-9f61-e858c580aeba service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:04:43.260948 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d8b46-ab91-4508-9f61-e858c580aeba service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:04:43.264556 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050847b-1f6a-4d17-8809-f1ce9ad87822 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:04:43.265191 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050847b-1f6a-4d17-8809-f1ce9ad87822 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:04:43.265475 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050847b-1f6a-4d17-8809-f1ce9ad87822 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:04:43.265755 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050847b-1f6a-4d17-8809-f1ce9ad87822 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:04:43.334126 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050847b-1f6a-4d17-8809-f1ce9ad87822 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:04:43.335956 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050847b-1f6a-4d17-8809-f1ce9ad87822 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:04:43.347073 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1343/2658] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:04:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:04:43.393840 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb6ca94-32a1-4f27-8652-ac59f55a1a6f service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:04:43.395443 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-cfb6ca94-32a1-4f27-8652-ac59f55a1a6f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:04:43.466076 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb6ca94-32a1-4f27-8652-ac59f55a1a6f service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:04:43.471526 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e766b-4d8f-425c-b45b-0f737df7d924 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:04:43.472202 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e766b-4d8f-425c-b45b-0f737df7d924 service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:04:43.472521 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e766b-4d8f-425c-b45b-0f737df7d924 service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:04:43.472803 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e766b-4d8f-425c-b45b-0f737df7d924 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:04:43.481419 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e766b-4d8f-425c-b45b-0f737df7d924 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:04:43.482983 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e766b-4d8f-425c-b45b-0f737df7d924 service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:04:43.491900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1316/2659] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:04:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:04:45.286569 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050847b-1f6a-4d17-8809-f1ce9ad87822 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:04:45.297899 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050847b-1f6a-4d17-8809-f1ce9ad87822 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:04:45.303560 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81014086-6240-499b-86bc-cf6abc2c286e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:04:45.304322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81014086-6240-499b-86bc-cf6abc2c286e service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:04:45.304644 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81014086-6240-499b-86bc-cf6abc2c286e service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:04:45.304924 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81014086-6240-499b-86bc-cf6abc2c286e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:04:45.306786 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-81014086-6240-499b-86bc-cf6abc2c286e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:04:45.385149 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81014086-6240-499b-86bc-cf6abc2c286e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:04:45.386723 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81014086-6240-499b-86bc-cf6abc2c286e service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:04:45.405144 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1344/2660] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:04:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:04:53.487703 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=34 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:04:55.402547 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=75 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:05:43.209536 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e766b-4d8f-425c-b45b-0f737df7d924 service nova] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:05:43.211085 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e02e766b-4d8f-425c-b45b-0f737df7d924 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:05:43.302128 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e766b-4d8f-425c-b45b-0f737df7d924 service nova] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:05:43.307151 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ca158-877a-46fd-a4cc-e9a2914afd1b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3560534427544cdabfd0ba702a0a165', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:05:43.308553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ca158-877a-46fd-a4cc-e9a2914afd1b service placement] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:05:43.308845 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ca158-877a-46fd-a4cc-e9a2914afd1b service placement] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:05:43.309125 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ca158-877a-46fd-a4cc-e9a2914afd1b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:05:43.310993 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-259ca158-877a-46fd-a4cc-e9a2914afd1b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:05:43.322257 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259ca158-877a-46fd-a4cc-e9a2914afd1b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:05:43.324520 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259ca158-877a-46fd-a4cc-e9a2914afd1b service placement] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:05:43.334630 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1317/2661] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:05:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:05:53.330459 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=40 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:07:43.734400 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81014086-6240-499b-86bc-cf6abc2c286e service nova] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:07:43.735956 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-81014086-6240-499b-86bc-cf6abc2c286e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:07:43.841654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81014086-6240-499b-86bc-cf6abc2c286e service nova] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:07:43.846069 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ac456-8a88-4473-acf2-21379037eca5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:07:43.846753 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ac456-8a88-4473-acf2-21379037eca5 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:07:43.847041 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ac456-8a88-4473-acf2-21379037eca5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:07:43.847321 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ac456-8a88-4473-acf2-21379037eca5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:07:43.849252 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-6b0ac456-8a88-4473-acf2-21379037eca5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:07:43.931795 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0ac456-8a88-4473-acf2-21379037eca5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:07:43.934459 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0ac456-8a88-4473-acf2-21379037eca5 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:07:43.943234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1345/2662] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:07:43 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:07:48.744546 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ca158-877a-46fd-a4cc-e9a2914afd1b service placement] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:07:48.744546 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-259ca158-877a-46fd-a4cc-e9a2914afd1b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:07:48.871459 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ca158-877a-46fd-a4cc-e9a2914afd1b service placement] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:07:48.876775 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feda153-188d-4b5b-b0a9-16f8695729f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:07:48.877641 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feda153-188d-4b5b-b0a9-16f8695729f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:07:48.877928 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feda153-188d-4b5b-b0a9-16f8695729f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:07:48.878202 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feda153-188d-4b5b-b0a9-16f8695729f4 admin admin] PATH_INFO: `/v3/users/e3ee9d190f9e45e2bc546d8b24f6568c` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:07:48.879379 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feda153-188d-4b5b-b0a9-16f8695729f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3ee9d190f9e45e2bc546d8b24f6568c)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:07:48.881328 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feda153-188d-4b5b-b0a9-16f8695729f4 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:07:49.045242 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1feda153-188d-4b5b-b0a9-16f8695729f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3ee9d190f9e45e2bc546d8b24f6568c'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:07:49.052596 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1feda153-188d-4b5b-b0a9-16f8695729f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3ee9d190f9e45e2bc546d8b24f6568c'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:07:49.064869 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1feda153-188d-4b5b-b0a9-16f8695729f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3ee9d190f9e45e2bc546d8b24f6568c'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:07:49.065580 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1feda153-188d-4b5b-b0a9-16f8695729f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3ee9d190f9e45e2bc546d8b24f6568c'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:07:49.066198 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1feda153-188d-4b5b-b0a9-16f8695729f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3ee9d190f9e45e2bc546d8b24f6568c'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:07:49.123727 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1318/2663] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 07:07:48 2022] DELETE /identity/v3/users/e3ee9d190f9e45e2bc546d8b24f6568c => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:07:49.128194 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769be1a-40df-46c5-83f4-2640d3fb96be None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:07:49.128725 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769be1a-40df-46c5-83f4-2640d3fb96be None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:07:49.128845 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769be1a-40df-46c5-83f4-2640d3fb96be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:07:49.155618 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-c769be1a-40df-46c5-83f4-2640d3fb96be None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:07:49.156869 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c769be1a-40df-46c5-83f4-2640d3fb96be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:07:49.197538 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:07:49.216917 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1346/2664] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:07:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:07:49.227481 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feda153-188d-4b5b-b0a9-16f8695729f4 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:07:49.228974 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-1feda153-188d-4b5b-b0a9-16f8695729f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:07:49.250031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:07:49.287238 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feda153-188d-4b5b-b0a9-16f8695729f4 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:07:49.290773 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8c358-2774-4972-a340-6b29572f6dfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:07:49.291411 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8c358-2774-4972-a340-6b29572f6dfd service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:07:49.291696 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8c358-2774-4972-a340-6b29572f6dfd service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:07:49.292071 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8c358-2774-4972-a340-6b29572f6dfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:07:49.301117 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8c358-2774-4972-a340-6b29572f6dfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:07:49.307803 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8c358-2774-4972-a340-6b29572f6dfd service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:07:49.312368 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1319/2665] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:07:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:07:49.691087 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769be1a-40df-46c5-83f4-2640d3fb96be None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:07:49.692631 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c769be1a-40df-46c5-83f4-2640d3fb96be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:07:49.707142 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769be1a-40df-46c5-83f4-2640d3fb96be None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:07:49.712057 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faebd3cf-5b0c-4667-bde2-c538f03396fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:07:49.712980 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faebd3cf-5b0c-4667-bde2-c538f03396fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:07:49.713255 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faebd3cf-5b0c-4667-bde2-c538f03396fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:07:49.713585 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faebd3cf-5b0c-4667-bde2-c538f03396fd admin admin] PATH_INFO: `/v3/projects/4f560c50833d4fb29fa6e0317eb96a47` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:07:49.714816 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faebd3cf-5b0c-4667-bde2-c538f03396fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f560c50833d4fb29fa6e0317eb96a47)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:07:49.717840 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faebd3cf-5b0c-4667-bde2-c538f03396fd admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:07:49.827203 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-faebd3cf-5b0c-4667-bde2-c538f03396fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f560c50833d4fb29fa6e0317eb96a47', 'request_id': 'req-faebd3cf-5b0c-4667-bde2-c538f03396fd'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:07:49.833712 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-faebd3cf-5b0c-4667-bde2-c538f03396fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f560c50833d4fb29fa6e0317eb96a47', 'request_id': 'req-faebd3cf-5b0c-4667-bde2-c538f03396fd'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:07:49.911659 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-faebd3cf-5b0c-4667-bde2-c538f03396fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f560c50833d4fb29fa6e0317eb96a47', 'request_id': 'req-faebd3cf-5b0c-4667-bde2-c538f03396fd'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:07:49.922478 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-faebd3cf-5b0c-4667-bde2-c538f03396fd admin admin] The token cache is being invalidate because project 4f560c50833d4fb29fa6e0317eb96a47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 07:07:49.924293 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-faebd3cf-5b0c-4667-bde2-c538f03396fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:07:49.924293 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-faebd3cf-5b0c-4667-bde2-c538f03396fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:07:49.925129 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1347/2666] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 07:07:49 2022] DELETE /identity/v3/projects/4f560c50833d4fb29fa6e0317eb96a47 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:07:49.929986 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2f593-dc55-4def-82aa-f4f21c083a21 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:07:49.930470 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2f593-dc55-4def-82aa-f4f21c083a21 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:07:49.930584 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2f593-dc55-4def-82aa-f4f21c083a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:07:49.977881 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-fbb2f593-dc55-4def-82aa-f4f21c083a21 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:07:49.979082 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-fbb2f593-dc55-4def-82aa-f4f21c083a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:07:50.037078 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1320/2667] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:07:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:07:50.045182 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfd5cb-0ff6-4430-97ad-f248d490531e None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:07:50.045643 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfd5cb-0ff6-4430-97ad-f248d490531e None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:07:50.046051 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfd5cb-0ff6-4430-97ad-f248d490531e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:07:50.064510 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-e0bfd5cb-0ff6-4430-97ad-f248d490531e None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:07:50.065854 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e0bfd5cb-0ff6-4430-97ad-f248d490531e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:07:50.073736 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1348/2668] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:07:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 07:07:50.082018 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb2f593-dc55-4def-82aa-f4f21c083a21 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:07:50.091755 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb2f593-dc55-4def-82aa-f4f21c083a21 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:07:50.096401 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f202b-237a-4063-8b5a-a83685c608ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:07:50.097122 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f202b-237a-4063-8b5a-a83685c608ed admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:07:50.097403 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f202b-237a-4063-8b5a-a83685c608ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:07:50.097684 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f202b-237a-4063-8b5a-a83685c608ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:07:50.098415 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7f202b-237a-4063-8b5a-a83685c608ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 07:07:50.098721 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7f202b-237a-4063-8b5a-a83685c608ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:07:50.100424 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7f202b-237a-4063-8b5a-a83685c608ed admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:07:50.112089 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1321/2669] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 07:07:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 07:07:50.116453 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bfd5cb-0ff6-4430-97ad-f248d490531e None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:07:50.126467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bfd5cb-0ff6-4430-97ad-f248d490531e None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:07:50.131286 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d135bfd-87f9-4f34-850c-20f57bf1c3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:07:50.137862 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d135bfd-87f9-4f34-850c-20f57bf1c3b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:07:50.137862 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d135bfd-87f9-4f34-850c-20f57bf1c3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:07:50.137862 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d135bfd-87f9-4f34-850c-20f57bf1c3b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:07:50.137862 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d135bfd-87f9-4f34-850c-20f57bf1c3b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:07:50.137862 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d135bfd-87f9-4f34-850c-20f57bf1c3b4 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:07:50.227041 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1349/2670] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 07:07:50 2022] POST /identity/v3/projects => generated 381 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 07:07:50.231275 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f202b-237a-4063-8b5a-a83685c608ed admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:07:50.242402 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f202b-237a-4063-8b5a-a83685c608ed admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:07:50.298285 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82adf02-148c-453a-a67f-7634e2a6721e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:07:50.298945 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82adf02-148c-453a-a67f-7634e2a6721e admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:07:50.299234 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82adf02-148c-453a-a67f-7634e2a6721e admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:07:50.299514 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82adf02-148c-453a-a67f-7634e2a6721e admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:07:50.300210 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82adf02-148c-453a-a67f-7634e2a6721e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:07:50.302173 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82adf02-148c-453a-a67f-7634e2a6721e admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:07:50.380808 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1322/2671] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 07:07:50 2022] POST /identity/v3/users => generated 291 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 07:07:50.384997 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d135bfd-87f9-4f34-850c-20f57bf1c3b4 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:07:50.395624 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d135bfd-87f9-4f34-850c-20f57bf1c3b4 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:07:50.400313 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b72be6-bc40-4208-82d7-cfe4d8132eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:07:50.401001 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b72be6-bc40-4208-82d7-cfe4d8132eb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:07:50.401293 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b72be6-bc40-4208-82d7-cfe4d8132eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:07:50.401578 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b72be6-bc40-4208-82d7-cfe4d8132eb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:07:50.402305 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b72be6-bc40-4208-82d7-cfe4d8132eb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:07:50.403903 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b72be6-bc40-4208-82d7-cfe4d8132eb5 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:07:50.413885 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1350/2672] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:07:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:07:50.423539 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82adf02-148c-453a-a67f-7634e2a6721e admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:07:50.434149 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82adf02-148c-453a-a67f-7634e2a6721e admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:07:50.439468 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf5bc5-4d5b-46a4-afa4-478110eec814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:07:50.440131 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf5bc5-4d5b-46a4-afa4-478110eec814 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:07:50.440433 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf5bc5-4d5b-46a4-afa4-478110eec814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:07:50.440716 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf5bc5-4d5b-46a4-afa4-478110eec814 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:07:50.441638 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf5bc5-4d5b-46a4-afa4-478110eec814 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:07:50.443219 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf5bc5-4d5b-46a4-afa4-478110eec814 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:07:50.456026 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1323/2673] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:07:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:07:50.460105 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b72be6-bc40-4208-82d7-cfe4d8132eb5 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:07:50.470259 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b72be6-bc40-4208-82d7-cfe4d8132eb5 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:07:50.474905 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8877ec3a-fb3b-4eee-8a95-b3a99c169c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:07:50.475771 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877ec3a-fb3b-4eee-8a95-b3a99c169c73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:07:50.476060 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877ec3a-fb3b-4eee-8a95-b3a99c169c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:07:50.476368 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877ec3a-fb3b-4eee-8a95-b3a99c169c73 admin admin] PATH_INFO: `/v3/projects/d9eda3f7979d459b86ead95d5d4eb2cd/users/4418e2afae0f4bdc9b918d68dcdf3dab/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:07:50.491487 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8877ec3a-fb3b-4eee-8a95-b3a99c169c73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9eda3f7979d459b86ead95d5d4eb2cd, user_id=4418e2afae0f4bdc9b918d68dcdf3dab, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:07:50.494189 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8877ec3a-fb3b-4eee-8a95-b3a99c169c73 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:07:50.568096 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1351/2674] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 07:07:50 2022] PUT /identity/v3/projects/d9eda3f7979d459b86ead95d5d4eb2cd/users/4418e2afae0f4bdc9b918d68dcdf3dab/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 07:07:50.578783 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddbe630-a7e5-4d58-b354-de3e3df1366d None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:07:50.579087 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddbe630-a7e5-4d58-b354-de3e3df1366d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:07:50.579403 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddbe630-a7e5-4d58-b354-de3e3df1366d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:07:50.585588 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:07:50.606333 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-7ddbe630-a7e5-4d58-b354-de3e3df1366d None None] MFA Rules not processed for user `4418e2afae0f4bdc9b918d68dcdf3dab`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:07:50.608086 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7ddbe630-a7e5-4d58-b354-de3e3df1366d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:07:50.673080 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1324/2675] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:07:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:07:50.677395 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8877ec3a-fb3b-4eee-8a95-b3a99c169c73 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:07:50.734896 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:07:50.739951 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8877ec3a-fb3b-4eee-8a95-b3a99c169c73 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:07:50.744595 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd0897-1bb0-457e-ba37-a7c5e8ebbcd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:07:50.745278 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd0897-1bb0-457e-ba37-a7c5e8ebbcd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:07:50.745567 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd0897-1bb0-457e-ba37-a7c5e8ebbcd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:07:50.745846 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd0897-1bb0-457e-ba37-a7c5e8ebbcd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:07:50.746530 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd0897-1bb0-457e-ba37-a7c5e8ebbcd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:07:50.748763 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd0897-1bb0-457e-ba37-a7c5e8ebbcd4 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:07:50.837268 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1352/2676] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 07:07:50 2022] POST /identity/v3/projects => generated 381 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 07:07:50.838421 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddbe630-a7e5-4d58-b354-de3e3df1366d None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:07:50.850013 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddbe630-a7e5-4d58-b354-de3e3df1366d None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:07:50.906840 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d4c6c-ec5f-4aa7-a849-464a900d1a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:07:50.909240 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d4c6c-ec5f-4aa7-a849-464a900d1a1e admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:07:50.909240 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d4c6c-ec5f-4aa7-a849-464a900d1a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:07:50.909240 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d4c6c-ec5f-4aa7-a849-464a900d1a1e admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:07:50.909240 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62d4c6c-ec5f-4aa7-a849-464a900d1a1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:07:50.910648 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62d4c6c-ec5f-4aa7-a849-464a900d1a1e admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:07:51.004132 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1325/2677] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 07:07:50 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 07:07:51.008016 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd0897-1bb0-457e-ba37-a7c5e8ebbcd4 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:07:51.018987 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd0897-1bb0-457e-ba37-a7c5e8ebbcd4 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:07:51.023621 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e81d8-5111-4cd6-bcb0-34772e2f7d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:07:51.024279 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e81d8-5111-4cd6-bcb0-34772e2f7d57 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:07:51.024600 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e81d8-5111-4cd6-bcb0-34772e2f7d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:07:51.024886 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e81d8-5111-4cd6-bcb0-34772e2f7d57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:07:51.025610 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e81d8-5111-4cd6-bcb0-34772e2f7d57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:07:51.027238 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e81d8-5111-4cd6-bcb0-34772e2f7d57 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:07:51.038492 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1353/2678] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:07:51.045236 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d4c6c-ec5f-4aa7-a849-464a900d1a1e admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:07:51.055543 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d4c6c-ec5f-4aa7-a849-464a900d1a1e admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:07:51.060332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c700f33-fb07-4bee-967b-21c263df52fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:07:51.061388 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c700f33-fb07-4bee-967b-21c263df52fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:07:51.061677 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c700f33-fb07-4bee-967b-21c263df52fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:07:51.061961 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c700f33-fb07-4bee-967b-21c263df52fa admin admin] PATH_INFO: `/v3/projects/06357308436d419e916d8be3671acd11/users/8357685dd61c4fc9a890fb9aa1181251/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:07:51.077308 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c700f33-fb07-4bee-967b-21c263df52fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=06357308436d419e916d8be3671acd11, user_id=8357685dd61c4fc9a890fb9aa1181251, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:07:51.080131 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c700f33-fb07-4bee-967b-21c263df52fa admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:07:51.138546 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1326/2679] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 07:07:51 2022] PUT /identity/v3/projects/06357308436d419e916d8be3671acd11/users/8357685dd61c4fc9a890fb9aa1181251/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:07:51.143669 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e81d8-5111-4cd6-bcb0-34772e2f7d57 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:07:51.155874 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e81d8-5111-4cd6-bcb0-34772e2f7d57 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:07:51.211341 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7123f0-9162-4e57-9298-356964e424af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:07:51.212025 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7123f0-9162-4e57-9298-356964e424af admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:07:51.212315 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7123f0-9162-4e57-9298-356964e424af admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:07:51.212628 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7123f0-9162-4e57-9298-356964e424af admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:07:51.213357 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7123f0-9162-4e57-9298-356964e424af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:07:51.214967 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7123f0-9162-4e57-9298-356964e424af admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:07:51.225467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1354/2680] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:07:51.229682 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c700f33-fb07-4bee-967b-21c263df52fa admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:07:51.239873 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c700f33-fb07-4bee-967b-21c263df52fa admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:07:51.244553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0070b52-9c5b-40c5-aa1f-0679a273c430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:07:51.245430 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0070b52-9c5b-40c5-aa1f-0679a273c430 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:07:51.245719 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0070b52-9c5b-40c5-aa1f-0679a273c430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:07:51.246000 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0070b52-9c5b-40c5-aa1f-0679a273c430 admin admin] PATH_INFO: `/v3/projects/06357308436d419e916d8be3671acd11/users/8357685dd61c4fc9a890fb9aa1181251/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:07:51.247984 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0070b52-9c5b-40c5-aa1f-0679a273c430 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06357308436d419e916d8be3671acd11, user_id=8357685dd61c4fc9a890fb9aa1181251, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:07:51.250779 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0070b52-9c5b-40c5-aa1f-0679a273c430 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:07:51.264627 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1327/2681] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 07:07:51 2022] PUT /identity/v3/projects/06357308436d419e916d8be3671acd11/users/8357685dd61c4fc9a890fb9aa1181251/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:07:51.275755 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61423b18-4fbf-49c8-becc-5962262a1ec4 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:07:51.276070 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61423b18-4fbf-49c8-becc-5962262a1ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:07:51.276352 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61423b18-4fbf-49c8-becc-5962262a1ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:07:51.303667 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-61423b18-4fbf-49c8-becc-5962262a1ec4 None None] MFA Rules not processed for user `8357685dd61c4fc9a890fb9aa1181251`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:07:51.304904 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-61423b18-4fbf-49c8-becc-5962262a1ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:07:51.371507 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1355/2682] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:07:51 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:07:51.383262 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0070b52-9c5b-40c5-aa1f-0679a273c430 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:07:51.384972 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b0070b52-9c5b-40c5-aa1f-0679a273c430 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:07:51.430491 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:07:51.441260 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0070b52-9c5b-40c5-aa1f-0679a273c430 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:07:51.445076 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa43e7-6e78-457e-a25a-ed066560b7cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:07:51.445713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa43e7-6e78-457e-a25a-ed066560b7cd service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:07:51.445997 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa43e7-6e78-457e-a25a-ed066560b7cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:07:51.446279 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa43e7-6e78-457e-a25a-ed066560b7cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:07:51.456391 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaa43e7-6e78-457e-a25a-ed066560b7cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:07:51.457845 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaa43e7-6e78-457e-a25a-ed066560b7cd service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:07:51.468281 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1328/2683] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:07:51 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:07:51.678948 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61423b18-4fbf-49c8-becc-5962262a1ec4 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:07:51.690033 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61423b18-4fbf-49c8-becc-5962262a1ec4 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:07:51.693638 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4efc18-8750-4fb0-8258-bd7a95a336ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:07:51.694309 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4efc18-8750-4fb0-8258-bd7a95a336ef service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:07:51.694659 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4efc18-8750-4fb0-8258-bd7a95a336ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:07:51.694969 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4efc18-8750-4fb0-8258-bd7a95a336ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:07:51.742760 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:07:51.753248 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4efc18-8750-4fb0-8258-bd7a95a336ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:07:51.754687 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4efc18-8750-4fb0-8258-bd7a95a336ef service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:07:51.763637 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1356/2684] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:07:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 07:07:54.572374 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a58976-6565-487e-bdcc-daf9e5865e38 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:07:54.572873 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a58976-6565-487e-bdcc-daf9e5865e38 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:07:54.572975 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a58976-6565-487e-bdcc-daf9e5865e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:07:54.621307 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-90a58976-6565-487e-bdcc-daf9e5865e38 None None] MFA Rules not processed for user `de31ee0904954d55b877e173ddc2c8be`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:07:54.622504 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-90a58976-6565-487e-bdcc-daf9e5865e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:07:54.688045 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1329/2685] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:07:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:07:54.937458 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4efc18-8750-4fb0-8258-bd7a95a336ef service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:07:54.938860 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ad4efc18-8750-4fb0-8258-bd7a95a336ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:07:55.006828 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4efc18-8750-4fb0-8258-bd7a95a336ef service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:07:55.010735 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c4524-2fcf-40a4-9244-67e31027780b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:07:55.011432 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c4524-2fcf-40a4-9244-67e31027780b service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:07:55.011720 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c4524-2fcf-40a4-9244-67e31027780b service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:07:55.012003 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c4524-2fcf-40a4-9244-67e31027780b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:07:55.014329 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d61c4524-2fcf-40a4-9244-67e31027780b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:07:55.024646 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c4524-2fcf-40a4-9244-67e31027780b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:07:55.026084 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c4524-2fcf-40a4-9244-67e31027780b service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:07:55.035324 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1357/2686] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:07:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:07:56.422056 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a58976-6565-487e-bdcc-daf9e5865e38 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:07:56.423458 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-90a58976-6565-487e-bdcc-daf9e5865e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:07:56.498383 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a58976-6565-487e-bdcc-daf9e5865e38 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:07:56.504861 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535d479-c9ab-4892-9e1a-fcd7d57281f5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3560534427544cdabfd0ba702a0a165', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:07:56.505527 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535d479-c9ab-4892-9e1a-fcd7d57281f5 service placement] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:07:56.505821 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535d479-c9ab-4892-9e1a-fcd7d57281f5 service placement] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:07:56.506105 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535d479-c9ab-4892-9e1a-fcd7d57281f5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:07:56.507958 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-d535d479-c9ab-4892-9e1a-fcd7d57281f5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:07:56.521016 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d535d479-c9ab-4892-9e1a-fcd7d57281f5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:07:56.522683 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d535d479-c9ab-4892-9e1a-fcd7d57281f5 service placement] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:07:56.547014 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1330/2687] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:07:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:08:03.279624 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c4524-2fcf-40a4-9244-67e31027780b service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:03.281622 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d61c4524-2fcf-40a4-9244-67e31027780b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:08:03.388489 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c4524-2fcf-40a4-9244-67e31027780b service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:03.394480 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98642424-70e9-4562-a594-c1a6fb879d38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:03.395149 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98642424-70e9-4562-a594-c1a6fb879d38 service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:03.395433 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98642424-70e9-4562-a594-c1a6fb879d38 service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:03.395733 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98642424-70e9-4562-a594-c1a6fb879d38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:03.397603 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-98642424-70e9-4562-a594-c1a6fb879d38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:08:03.414716 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98642424-70e9-4562-a594-c1a6fb879d38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:03.416446 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98642424-70e9-4562-a594-c1a6fb879d38 service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:03.428368 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1358/2688] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:08:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:08:05.478740 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535d479-c9ab-4892-9e1a-fcd7d57281f5 service placement] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:05.540268 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:08:05.557718 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535d479-c9ab-4892-9e1a-fcd7d57281f5 service placement] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:05.562806 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f6283-dc29-4724-bb1a-005ecfbb8d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:05.563762 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f6283-dc29-4724-bb1a-005ecfbb8d4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:05.564120 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f6283-dc29-4724-bb1a-005ecfbb8d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:05.564518 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f6283-dc29-4724-bb1a-005ecfbb8d4f admin admin] PATH_INFO: `/v3/users/4418e2afae0f4bdc9b918d68dcdf3dab` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:05.565790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f6283-dc29-4724-bb1a-005ecfbb8d4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4418e2afae0f4bdc9b918d68dcdf3dab)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:05.567773 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f6283-dc29-4724-bb1a-005ecfbb8d4f admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:05.723113 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e37f6283-dc29-4724-bb1a-005ecfbb8d4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4418e2afae0f4bdc9b918d68dcdf3dab'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:05.730178 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e37f6283-dc29-4724-bb1a-005ecfbb8d4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4418e2afae0f4bdc9b918d68dcdf3dab'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:05.742030 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e37f6283-dc29-4724-bb1a-005ecfbb8d4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4418e2afae0f4bdc9b918d68dcdf3dab'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:05.742745 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e37f6283-dc29-4724-bb1a-005ecfbb8d4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4418e2afae0f4bdc9b918d68dcdf3dab'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:05.743343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e37f6283-dc29-4724-bb1a-005ecfbb8d4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4418e2afae0f4bdc9b918d68dcdf3dab'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:05.830959 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1331/2689] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 07:08:05 2022] DELETE /identity/v3/users/4418e2afae0f4bdc9b918d68dcdf3dab => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:08:05.835335 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7726a-49f9-42c1-bfe6-4dc4855b0910 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:05.839248 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7726a-49f9-42c1-bfe6-4dc4855b0910 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:05.839248 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7726a-49f9-42c1-bfe6-4dc4855b0910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:05.853592 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-3ba7726a-49f9-42c1-bfe6-4dc4855b0910 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:08:05.854795 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-3ba7726a-49f9-42c1-bfe6-4dc4855b0910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:08:05.900658 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:08:05.914453 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1359/2690] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:08:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:08:05.924172 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f6283-dc29-4724-bb1a-005ecfbb8d4f admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:05.925664 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e37f6283-dc29-4724-bb1a-005ecfbb8d4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:08:05.983938 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f6283-dc29-4724-bb1a-005ecfbb8d4f admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:05.987543 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fec98e-abfe-4904-b29b-1c8ba60ab83f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:05.988188 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fec98e-abfe-4904-b29b-1c8ba60ab83f service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:05.988524 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fec98e-abfe-4904-b29b-1c8ba60ab83f service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:05.988811 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fec98e-abfe-4904-b29b-1c8ba60ab83f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:05.997803 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fec98e-abfe-4904-b29b-1c8ba60ab83f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:05.999863 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fec98e-abfe-4904-b29b-1c8ba60ab83f service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:06.009337 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1332/2691] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:08:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:08:06.422410 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba7726a-49f9-42c1-bfe6-4dc4855b0910 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:06.423826 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-3ba7726a-49f9-42c1-bfe6-4dc4855b0910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:08:06.443281 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba7726a-49f9-42c1-bfe6-4dc4855b0910 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:06.449213 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d13ec-60ed-4bde-81fb-38227031ec16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:06.450168 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d13ec-60ed-4bde-81fb-38227031ec16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:06.450503 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d13ec-60ed-4bde-81fb-38227031ec16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:06.450650 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d13ec-60ed-4bde-81fb-38227031ec16 admin admin] PATH_INFO: `/v3/projects/d9eda3f7979d459b86ead95d5d4eb2cd` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:06.451896 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718d13ec-60ed-4bde-81fb-38227031ec16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9eda3f7979d459b86ead95d5d4eb2cd)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:06.453906 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718d13ec-60ed-4bde-81fb-38227031ec16 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:06.593460 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-718d13ec-60ed-4bde-81fb-38227031ec16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9eda3f7979d459b86ead95d5d4eb2cd', 'request_id': 'req-718d13ec-60ed-4bde-81fb-38227031ec16'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:06.611033 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-718d13ec-60ed-4bde-81fb-38227031ec16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9eda3f7979d459b86ead95d5d4eb2cd', 'request_id': 'req-718d13ec-60ed-4bde-81fb-38227031ec16'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:06.692825 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-718d13ec-60ed-4bde-81fb-38227031ec16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9eda3f7979d459b86ead95d5d4eb2cd', 'request_id': 'req-718d13ec-60ed-4bde-81fb-38227031ec16'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:06.704968 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-718d13ec-60ed-4bde-81fb-38227031ec16 admin admin] The token cache is being invalidate because project d9eda3f7979d459b86ead95d5d4eb2cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 07:08:06.705265 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-718d13ec-60ed-4bde-81fb-38227031ec16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:06.706331 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-718d13ec-60ed-4bde-81fb-38227031ec16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:06.708252 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1360/2692] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 07:08:06 2022] DELETE /identity/v3/projects/d9eda3f7979d459b86ead95d5d4eb2cd => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:08:06.712242 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fec98e-abfe-4904-b29b-1c8ba60ab83f service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:06.723965 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fec98e-abfe-4904-b29b-1c8ba60ab83f service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:06.785904 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8fecf-0107-4e66-ace6-8d4cd5afbcf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:06.786778 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8fecf-0107-4e66-ace6-8d4cd5afbcf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:06.787064 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8fecf-0107-4e66-ace6-8d4cd5afbcf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:06.787345 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8fecf-0107-4e66-ace6-8d4cd5afbcf6 admin admin] PATH_INFO: `/v3/users/8357685dd61c4fc9a890fb9aa1181251` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:06.788588 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa8fecf-0107-4e66-ace6-8d4cd5afbcf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8357685dd61c4fc9a890fb9aa1181251)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:06.790551 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa8fecf-0107-4e66-ace6-8d4cd5afbcf6 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:06.800702 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:08:07.005429 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-8aa8fecf-0107-4e66-ace6-8d4cd5afbcf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8357685dd61c4fc9a890fb9aa1181251'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:07.012059 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-8aa8fecf-0107-4e66-ace6-8d4cd5afbcf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8357685dd61c4fc9a890fb9aa1181251'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:07.024729 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-8aa8fecf-0107-4e66-ace6-8d4cd5afbcf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8357685dd61c4fc9a890fb9aa1181251'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:07.025423 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-8aa8fecf-0107-4e66-ace6-8d4cd5afbcf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8357685dd61c4fc9a890fb9aa1181251'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:07.026041 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-8aa8fecf-0107-4e66-ace6-8d4cd5afbcf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8357685dd61c4fc9a890fb9aa1181251'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:07.103793 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1333/2693] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 07:08:06 2022] DELETE /identity/v3/users/8357685dd61c4fc9a890fb9aa1181251 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:08:07.513159 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d13ec-60ed-4bde-81fb-38227031ec16 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:07.515300 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-718d13ec-60ed-4bde-81fb-38227031ec16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:08:07.531471 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d13ec-60ed-4bde-81fb-38227031ec16 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:07.592845 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7748b57-ddfe-44e3-9ac0-35ecd0bf9a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:07.593728 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7748b57-ddfe-44e3-9ac0-35ecd0bf9a85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:07.594017 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7748b57-ddfe-44e3-9ac0-35ecd0bf9a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:07.594305 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7748b57-ddfe-44e3-9ac0-35ecd0bf9a85 admin admin] PATH_INFO: `/v3/projects/06357308436d419e916d8be3671acd11` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:07.595499 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7748b57-ddfe-44e3-9ac0-35ecd0bf9a85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06357308436d419e916d8be3671acd11)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:07.597452 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7748b57-ddfe-44e3-9ac0-35ecd0bf9a85 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:07.708656 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e7748b57-ddfe-44e3-9ac0-35ecd0bf9a85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06357308436d419e916d8be3671acd11', 'request_id': 'req-e7748b57-ddfe-44e3-9ac0-35ecd0bf9a85'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:07.715025 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e7748b57-ddfe-44e3-9ac0-35ecd0bf9a85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06357308436d419e916d8be3671acd11', 'request_id': 'req-e7748b57-ddfe-44e3-9ac0-35ecd0bf9a85'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:07.795668 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e7748b57-ddfe-44e3-9ac0-35ecd0bf9a85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06357308436d419e916d8be3671acd11', 'request_id': 'req-e7748b57-ddfe-44e3-9ac0-35ecd0bf9a85'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:07.807323 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e7748b57-ddfe-44e3-9ac0-35ecd0bf9a85 admin admin] The token cache is being invalidate because project 06357308436d419e916d8be3671acd11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 07:08:07.807618 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e7748b57-ddfe-44e3-9ac0-35ecd0bf9a85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:07.808264 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e7748b57-ddfe-44e3-9ac0-35ecd0bf9a85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:07.810112 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1361/2694] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 07:08:07 2022] DELETE /identity/v3/projects/06357308436d419e916d8be3671acd11 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:08:07.815773 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47df28d-7b4f-43c5-b4b7-db4e6922156c None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:07.816263 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47df28d-7b4f-43c5-b4b7-db4e6922156c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:07.816438 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47df28d-7b4f-43c5-b4b7-db4e6922156c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:07.835096 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-c47df28d-7b4f-43c5-b4b7-db4e6922156c None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:08:07.836364 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-c47df28d-7b4f-43c5-b4b7-db4e6922156c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:08:07.909153 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1334/2695] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:08:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:08:07.916433 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27db15-cbcd-4513-81fc-2ba2769a7678 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:07.916791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27db15-cbcd-4513-81fc-2ba2769a7678 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:07.917082 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27db15-cbcd-4513-81fc-2ba2769a7678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:07.925697 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:08:07.941174 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-be27db15-cbcd-4513-81fc-2ba2769a7678 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:08:07.942347 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-be27db15-cbcd-4513-81fc-2ba2769a7678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:08:07.951278 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1362/2696] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:08:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:08:07.956158 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47df28d-7b4f-43c5-b4b7-db4e6922156c None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:07.965952 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47df28d-7b4f-43c5-b4b7-db4e6922156c None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:07.971367 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12151ff-46cb-4a3f-9673-adf295a95173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:07.972031 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12151ff-46cb-4a3f-9673-adf295a95173 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:07.972317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12151ff-46cb-4a3f-9673-adf295a95173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:07.972628 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12151ff-46cb-4a3f-9673-adf295a95173 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:07.973421 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12151ff-46cb-4a3f-9673-adf295a95173 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 07:08:07.973718 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12151ff-46cb-4a3f-9673-adf295a95173 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:07.975315 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12151ff-46cb-4a3f-9673-adf295a95173 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:07.988252 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1335/2697] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 07:08:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 07:08:07.992541 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be27db15-cbcd-4513-81fc-2ba2769a7678 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:08.002098 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be27db15-cbcd-4513-81fc-2ba2769a7678 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:08.006835 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ea3c2-add6-42a7-a7fa-e54933f15a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:08.007539 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ea3c2-add6-42a7-a7fa-e54933f15a50 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:08.007822 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ea3c2-add6-42a7-a7fa-e54933f15a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:08.008102 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ea3c2-add6-42a7-a7fa-e54933f15a50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:08.008851 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ea3c2-add6-42a7-a7fa-e54933f15a50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:08.010747 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ea3c2-add6-42a7-a7fa-e54933f15a50 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:08.102387 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1363/2698] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 07:08:07 2022] POST /identity/v3/projects => generated 369 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 07:08:08.106094 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12151ff-46cb-4a3f-9673-adf295a95173 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:08.116519 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12151ff-46cb-4a3f-9673-adf295a95173 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:08.172890 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c7787-f6d1-4030-b3a2-cd45dca89aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:08.173540 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c7787-f6d1-4030-b3a2-cd45dca89aa6 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:08.173828 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c7787-f6d1-4030-b3a2-cd45dca89aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:08.174109 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c7787-f6d1-4030-b3a2-cd45dca89aa6 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:08.174814 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c7787-f6d1-4030-b3a2-cd45dca89aa6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:08.176789 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c7787-f6d1-4030-b3a2-cd45dca89aa6 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:08.266920 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1336/2699] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 07:08:08 2022] POST /identity/v3/users => generated 285 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 07:08:08.271353 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ea3c2-add6-42a7-a7fa-e54933f15a50 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:08.282687 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ea3c2-add6-42a7-a7fa-e54933f15a50 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:08.287498 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31110290-760c-4ebc-8a63-8102726d3c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:08.288105 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31110290-760c-4ebc-8a63-8102726d3c3a admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:08.288396 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31110290-760c-4ebc-8a63-8102726d3c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:08.288712 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31110290-760c-4ebc-8a63-8102726d3c3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:08.289447 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31110290-760c-4ebc-8a63-8102726d3c3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:08.291055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31110290-760c-4ebc-8a63-8102726d3c3a admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:08.301124 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1364/2700] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:08:08.305073 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c7787-f6d1-4030-b3a2-cd45dca89aa6 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:08.311511 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:08:08.315361 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c7787-f6d1-4030-b3a2-cd45dca89aa6 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:08.320026 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c7577a-7b37-4337-9856-dd67fc27a538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:08.320712 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7577a-7b37-4337-9856-dd67fc27a538 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:08.321004 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7577a-7b37-4337-9856-dd67fc27a538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:08.321289 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7577a-7b37-4337-9856-dd67fc27a538 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:08.322021 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c7577a-7b37-4337-9856-dd67fc27a538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:08.323642 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c7577a-7b37-4337-9856-dd67fc27a538 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:08.335011 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1337/2701] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:08:08.338972 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31110290-760c-4ebc-8a63-8102726d3c3a admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:08.349615 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31110290-760c-4ebc-8a63-8102726d3c3a admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:08.354214 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b77838e-c997-4b23-b9b4-86bc29d2a927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:08.355093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77838e-c997-4b23-b9b4-86bc29d2a927 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:08.355398 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77838e-c997-4b23-b9b4-86bc29d2a927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:08.355658 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77838e-c997-4b23-b9b4-86bc29d2a927 admin admin] PATH_INFO: `/v3/projects/d9903b0e79b14581befb665a8fe5547f/users/325f90ae3f7d4e908802bb096b24e367/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:08.371383 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b77838e-c997-4b23-b9b4-86bc29d2a927 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9903b0e79b14581befb665a8fe5547f, user_id=325f90ae3f7d4e908802bb096b24e367, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:08.374220 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b77838e-c997-4b23-b9b4-86bc29d2a927 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:08.460799 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1365/2702] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 07:08:08 2022] PUT /identity/v3/projects/d9903b0e79b14581befb665a8fe5547f/users/325f90ae3f7d4e908802bb096b24e367/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:08:08.471404 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06543382-59a4-4003-a3f2-3be07af0233a None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:08.471701 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06543382-59a4-4003-a3f2-3be07af0233a None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:08.472009 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06543382-59a4-4003-a3f2-3be07af0233a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:08.498071 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-06543382-59a4-4003-a3f2-3be07af0233a None None] MFA Rules not processed for user `325f90ae3f7d4e908802bb096b24e367`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:08:08.499276 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-06543382-59a4-4003-a3f2-3be07af0233a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:08:08.564035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1338/2703] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:08:08 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:08:08.568448 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b77838e-c997-4b23-b9b4-86bc29d2a927 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:08.630477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b77838e-c997-4b23-b9b4-86bc29d2a927 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:08.635368 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bedf71-5027-41d7-ad03-b6c4f94ab9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:08.636035 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bedf71-5027-41d7-ad03-b6c4f94ab9e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:08.636323 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bedf71-5027-41d7-ad03-b6c4f94ab9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:08.636625 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bedf71-5027-41d7-ad03-b6c4f94ab9e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:08.637326 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bedf71-5027-41d7-ad03-b6c4f94ab9e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:08.639882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bedf71-5027-41d7-ad03-b6c4f94ab9e3 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:08.740004 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1366/2704] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 07:08:08 2022] POST /identity/v3/projects => generated 367 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 07:08:08.744268 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06543382-59a4-4003-a3f2-3be07af0233a None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:08.759214 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06543382-59a4-4003-a3f2-3be07af0233a None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:08.822566 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580ac25-6568-457b-abe5-c864850994b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:08.823256 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580ac25-6568-457b-abe5-c864850994b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:08.823423 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580ac25-6568-457b-abe5-c864850994b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:08.823705 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580ac25-6568-457b-abe5-c864850994b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:08.824382 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2580ac25-6568-457b-abe5-c864850994b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:08.827106 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2580ac25-6568-457b-abe5-c864850994b1 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:08.928912 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1339/2705] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 07:08:08 2022] POST /identity/v3/users => generated 290 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 07:08:08.932976 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bedf71-5027-41d7-ad03-b6c4f94ab9e3 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:08.944398 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bedf71-5027-41d7-ad03-b6c4f94ab9e3 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:08.949253 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf4b2fb-4860-4cad-afad-7c2484f5de5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:08.950120 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf4b2fb-4860-4cad-afad-7c2484f5de5c admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:08.950227 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf4b2fb-4860-4cad-afad-7c2484f5de5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:08.950467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf4b2fb-4860-4cad-afad-7c2484f5de5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:08.951230 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf4b2fb-4860-4cad-afad-7c2484f5de5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:08.953390 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf4b2fb-4860-4cad-afad-7c2484f5de5c admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:08.965184 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1367/2706] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:08:08.969247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580ac25-6568-457b-abe5-c864850994b1 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:08.979398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580ac25-6568-457b-abe5-c864850994b1 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:08.984591 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9934d2b-d841-4254-80f8-84647c4652f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:08.985483 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9934d2b-d841-4254-80f8-84647c4652f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:08.985772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9934d2b-d841-4254-80f8-84647c4652f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:08.986054 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9934d2b-d841-4254-80f8-84647c4652f2 admin admin] PATH_INFO: `/v3/projects/f37bd254b8bc47f19f3f1ade3a582c0d/users/951882d379524e659f413dc3080a989a/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:09.001887 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9934d2b-d841-4254-80f8-84647c4652f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f37bd254b8bc47f19f3f1ade3a582c0d, user_id=951882d379524e659f413dc3080a989a, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:09.004590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9934d2b-d841-4254-80f8-84647c4652f2 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:09.100942 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1340/2707] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 07:08:08 2022] PUT /identity/v3/projects/f37bd254b8bc47f19f3f1ade3a582c0d/users/951882d379524e659f413dc3080a989a/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:08:09.104905 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf4b2fb-4860-4cad-afad-7c2484f5de5c admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:09.116063 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf4b2fb-4860-4cad-afad-7c2484f5de5c admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:09.172619 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b710d-d63f-4d8d-9c9a-9d2250b489ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:09.173299 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b710d-d63f-4d8d-9c9a-9d2250b489ac admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:09.173620 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b710d-d63f-4d8d-9c9a-9d2250b489ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:09.173903 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b710d-d63f-4d8d-9c9a-9d2250b489ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:09.174636 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b710d-d63f-4d8d-9c9a-9d2250b489ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:09.176242 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b710d-d63f-4d8d-9c9a-9d2250b489ac admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:09.186723 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1368/2708] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:08:09 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:08:09.190997 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9934d2b-d841-4254-80f8-84647c4652f2 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:09.201429 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9934d2b-d841-4254-80f8-84647c4652f2 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:09.206352 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2535ade5-9417-4c53-ba86-43fd226daa14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:09.207299 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535ade5-9417-4c53-ba86-43fd226daa14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:09.207587 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535ade5-9417-4c53-ba86-43fd226daa14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:09.207867 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535ade5-9417-4c53-ba86-43fd226daa14 admin admin] PATH_INFO: `/v3/projects/f37bd254b8bc47f19f3f1ade3a582c0d/users/951882d379524e659f413dc3080a989a/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:09.209888 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2535ade5-9417-4c53-ba86-43fd226daa14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f37bd254b8bc47f19f3f1ade3a582c0d, user_id=951882d379524e659f413dc3080a989a, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:09.212591 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2535ade5-9417-4c53-ba86-43fd226daa14 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:09.223297 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1341/2709] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 07:08:09 2022] PUT /identity/v3/projects/f37bd254b8bc47f19f3f1ade3a582c0d/users/951882d379524e659f413dc3080a989a/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:08:09.235467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf4f16-db6f-4fcd-96bf-cdde767f6bc0 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:09.235752 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf4f16-db6f-4fcd-96bf-cdde767f6bc0 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:09.236038 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf4f16-db6f-4fcd-96bf-cdde767f6bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:09.255740 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:08:09.263117 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-f9cf4f16-db6f-4fcd-96bf-cdde767f6bc0 None None] MFA Rules not processed for user `951882d379524e659f413dc3080a989a`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:08:09.264454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-f9cf4f16-db6f-4fcd-96bf-cdde767f6bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:08:09.338817 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1369/2710] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:08:09 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 07:08:09.349992 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2535ade5-9417-4c53-ba86-43fd226daa14 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:09.351800 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-2535ade5-9417-4c53-ba86-43fd226daa14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:08:09.393048 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:08:09.411070 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2535ade5-9417-4c53-ba86-43fd226daa14 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:09.414819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2842ec-09e7-44e2-a4d0-a8513521cdf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:09.415482 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2842ec-09e7-44e2-a4d0-a8513521cdf2 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:09.415774 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2842ec-09e7-44e2-a4d0-a8513521cdf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:09.416056 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2842ec-09e7-44e2-a4d0-a8513521cdf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:09.425975 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2842ec-09e7-44e2-a4d0-a8513521cdf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:09.427401 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2842ec-09e7-44e2-a4d0-a8513521cdf2 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:09.436907 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1342/2711] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:08:09 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:08:09.566140 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf4f16-db6f-4fcd-96bf-cdde767f6bc0 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:09.567539 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-f9cf4f16-db6f-4fcd-96bf-cdde767f6bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:08:09.579800 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf4f16-db6f-4fcd-96bf-cdde767f6bc0 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:09.589167 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ba7d4-58a6-465b-ba80-8203117f6384 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:09.589834 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ba7d4-58a6-465b-ba80-8203117f6384 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:09.590130 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ba7d4-58a6-465b-ba80-8203117f6384 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:09.590416 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ba7d4-58a6-465b-ba80-8203117f6384 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:09.646377 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ba7d4-58a6-465b-ba80-8203117f6384 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:09.647796 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ba7d4-58a6-465b-ba80-8203117f6384 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:09.657346 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1370/2712] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:08:09 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 07:08:10.231732 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2842ec-09e7-44e2-a4d0-a8513521cdf2 service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:10.233872 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-1d2842ec-09e7-44e2-a4d0-a8513521cdf2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:08:10.328600 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2842ec-09e7-44e2-a4d0-a8513521cdf2 service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:10.334875 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df172ab5-d99e-49d6-9b91-6982e886e415 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:10.335539 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df172ab5-d99e-49d6-9b91-6982e886e415 service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:10.335830 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df172ab5-d99e-49d6-9b91-6982e886e415 service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:10.336113 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df172ab5-d99e-49d6-9b91-6982e886e415 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:10.337960 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-df172ab5-d99e-49d6-9b91-6982e886e415 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:08:10.350788 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df172ab5-d99e-49d6-9b91-6982e886e415 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:10.357903 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df172ab5-d99e-49d6-9b91-6982e886e415 service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:10.368402 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1343/2713] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:08:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:08:14.631258 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ba7d4-58a6-465b-ba80-8203117f6384 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:14.704481 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ba7d4-58a6-465b-ba80-8203117f6384 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:14.709225 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517c683-e6ee-4da6-b283-a0fec9292d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:14.710095 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517c683-e6ee-4da6-b283-a0fec9292d29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:14.710379 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517c683-e6ee-4da6-b283-a0fec9292d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:14.710657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517c683-e6ee-4da6-b283-a0fec9292d29 admin admin] PATH_INFO: `/v3/users/325f90ae3f7d4e908802bb096b24e367` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:14.711842 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d517c683-e6ee-4da6-b283-a0fec9292d29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=325f90ae3f7d4e908802bb096b24e367)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:14.713772 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d517c683-e6ee-4da6-b283-a0fec9292d29 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:14.915482 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:08:14.924712 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d517c683-e6ee-4da6-b283-a0fec9292d29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '325f90ae3f7d4e908802bb096b24e367'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:14.931555 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d517c683-e6ee-4da6-b283-a0fec9292d29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '325f90ae3f7d4e908802bb096b24e367'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:14.932222 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d517c683-e6ee-4da6-b283-a0fec9292d29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '325f90ae3f7d4e908802bb096b24e367'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:14.932876 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d517c683-e6ee-4da6-b283-a0fec9292d29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '325f90ae3f7d4e908802bb096b24e367'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:14.944836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d517c683-e6ee-4da6-b283-a0fec9292d29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '325f90ae3f7d4e908802bb096b24e367'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:15.029711 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1371/2714] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 07:08:14 2022] DELETE /identity/v3/users/325f90ae3f7d4e908802bb096b24e367 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:08:15.035059 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f50af-fefe-47e0-91d4-c1b805471f66 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:15.035551 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f50af-fefe-47e0-91d4-c1b805471f66 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:15.035664 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f50af-fefe-47e0-91d4-c1b805471f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:15.054890 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-d68f50af-fefe-47e0-91d4-c1b805471f66 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:08:15.057229 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-d68f50af-fefe-47e0-91d4-c1b805471f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:08:15.121504 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1344/2715] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:08:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:08:15.131126 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517c683-e6ee-4da6-b283-a0fec9292d29 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:15.132537 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d517c683-e6ee-4da6-b283-a0fec9292d29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:08:15.190381 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517c683-e6ee-4da6-b283-a0fec9292d29 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:15.194072 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a68a4c-3376-4c5a-b055-4648586fb92c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:15.194723 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a68a4c-3376-4c5a-b055-4648586fb92c service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:15.195009 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a68a4c-3376-4c5a-b055-4648586fb92c service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:15.195285 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a68a4c-3376-4c5a-b055-4648586fb92c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:15.205087 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a68a4c-3376-4c5a-b055-4648586fb92c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:15.206877 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a68a4c-3376-4c5a-b055-4648586fb92c service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:15.216504 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1372/2716] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:08:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:08:15.628170 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f50af-fefe-47e0-91d4-c1b805471f66 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:15.629652 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-d68f50af-fefe-47e0-91d4-c1b805471f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:08:15.645449 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f50af-fefe-47e0-91d4-c1b805471f66 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:15.650724 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abbbeb3-551e-49b1-a7b0-d395f2290181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:15.651728 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abbbeb3-551e-49b1-a7b0-d395f2290181 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:15.652016 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abbbeb3-551e-49b1-a7b0-d395f2290181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:15.652298 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abbbeb3-551e-49b1-a7b0-d395f2290181 admin admin] PATH_INFO: `/v3/projects/d9903b0e79b14581befb665a8fe5547f` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:15.653811 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abbbeb3-551e-49b1-a7b0-d395f2290181 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9903b0e79b14581befb665a8fe5547f)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:15.656192 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abbbeb3-551e-49b1-a7b0-d395f2290181 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:15.765510 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-4abbbeb3-551e-49b1-a7b0-d395f2290181 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9903b0e79b14581befb665a8fe5547f', 'request_id': 'req-4abbbeb3-551e-49b1-a7b0-d395f2290181'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:15.771681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-4abbbeb3-551e-49b1-a7b0-d395f2290181 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9903b0e79b14581befb665a8fe5547f', 'request_id': 'req-4abbbeb3-551e-49b1-a7b0-d395f2290181'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:15.773549 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:08:15.849924 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-4abbbeb3-551e-49b1-a7b0-d395f2290181 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9903b0e79b14581befb665a8fe5547f', 'request_id': 'req-4abbbeb3-551e-49b1-a7b0-d395f2290181'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:15.864002 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-4abbbeb3-551e-49b1-a7b0-d395f2290181 admin admin] The token cache is being invalidate because project d9903b0e79b14581befb665a8fe5547f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 07:08:15.864320 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-4abbbeb3-551e-49b1-a7b0-d395f2290181 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:15.864978 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-4abbbeb3-551e-49b1-a7b0-d395f2290181 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:15.869562 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1345/2717] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 07:08:15 2022] DELETE /identity/v3/projects/d9903b0e79b14581befb665a8fe5547f => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:08:15.870716 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a68a4c-3376-4c5a-b055-4648586fb92c service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:15.882380 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a68a4c-3376-4c5a-b055-4648586fb92c service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:15.942070 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b3754-b24e-499e-a04c-8f6f63cadeab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:15.942960 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b3754-b24e-499e-a04c-8f6f63cadeab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:15.943250 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b3754-b24e-499e-a04c-8f6f63cadeab admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:15.943532 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b3754-b24e-499e-a04c-8f6f63cadeab admin admin] PATH_INFO: `/v3/users/951882d379524e659f413dc3080a989a` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:15.944771 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97b3754-b24e-499e-a04c-8f6f63cadeab admin admin] RBAC: Authorizing `identity:delete_user(user_id=951882d379524e659f413dc3080a989a)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:15.946715 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97b3754-b24e-499e-a04c-8f6f63cadeab admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:16.129910 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d97b3754-b24e-499e-a04c-8f6f63cadeab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '951882d379524e659f413dc3080a989a'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:16.136908 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d97b3754-b24e-499e-a04c-8f6f63cadeab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '951882d379524e659f413dc3080a989a'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:16.137584 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d97b3754-b24e-499e-a04c-8f6f63cadeab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '951882d379524e659f413dc3080a989a'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:16.138236 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d97b3754-b24e-499e-a04c-8f6f63cadeab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '951882d379524e659f413dc3080a989a'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:16.147078 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:08:16.158376 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d97b3754-b24e-499e-a04c-8f6f63cadeab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '951882d379524e659f413dc3080a989a'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:16.237746 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1373/2718] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 07:08:15 2022] DELETE /identity/v3/users/951882d379524e659f413dc3080a989a => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:08:16.659545 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abbbeb3-551e-49b1-a7b0-d395f2290181 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:16.661130 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4abbbeb3-551e-49b1-a7b0-d395f2290181 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:08:16.675336 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abbbeb3-551e-49b1-a7b0-d395f2290181 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:16.733320 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98804116-2492-4218-812b-d7ee661e6ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:16.734814 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98804116-2492-4218-812b-d7ee661e6ded admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:16.735277 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98804116-2492-4218-812b-d7ee661e6ded admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:16.735562 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98804116-2492-4218-812b-d7ee661e6ded admin admin] PATH_INFO: `/v3/projects/f37bd254b8bc47f19f3f1ade3a582c0d` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:16.736782 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98804116-2492-4218-812b-d7ee661e6ded admin admin] RBAC: Authorizing `identity:delete_project(project_id=f37bd254b8bc47f19f3f1ade3a582c0d)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:16.738790 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98804116-2492-4218-812b-d7ee661e6ded admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:16.878326 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-98804116-2492-4218-812b-d7ee661e6ded admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f37bd254b8bc47f19f3f1ade3a582c0d', 'request_id': 'req-98804116-2492-4218-812b-d7ee661e6ded'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:16.884068 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-98804116-2492-4218-812b-d7ee661e6ded admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f37bd254b8bc47f19f3f1ade3a582c0d', 'request_id': 'req-98804116-2492-4218-812b-d7ee661e6ded'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:16.969565 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-98804116-2492-4218-812b-d7ee661e6ded admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f37bd254b8bc47f19f3f1ade3a582c0d', 'request_id': 'req-98804116-2492-4218-812b-d7ee661e6ded'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:16.980913 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-98804116-2492-4218-812b-d7ee661e6ded admin admin] The token cache is being invalidate because project f37bd254b8bc47f19f3f1ade3a582c0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 07:08:16.981209 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-98804116-2492-4218-812b-d7ee661e6ded admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:16.981841 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-98804116-2492-4218-812b-d7ee661e6ded admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:16.983391 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1346/2719] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 07:08:16 2022] DELETE /identity/v3/projects/f37bd254b8bc47f19f3f1ade3a582c0d => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:08:16.988877 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af0b70-2a8e-4d1d-bbb3-5e70483b788a None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:16.989385 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af0b70-2a8e-4d1d-bbb3-5e70483b788a None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:16.989514 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af0b70-2a8e-4d1d-bbb3-5e70483b788a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:17.009254 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-08af0b70-2a8e-4d1d-bbb3-5e70483b788a None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:08:17.010442 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-08af0b70-2a8e-4d1d-bbb3-5e70483b788a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:08:17.069793 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1374/2720] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:08:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:08:17.077406 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a3094-dd9b-4f85-acac-6de032854974 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:17.077704 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a3094-dd9b-4f85-acac-6de032854974 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:17.077987 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a3094-dd9b-4f85-acac-6de032854974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:17.097340 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-3f3a3094-dd9b-4f85-acac-6de032854974 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:08:17.098539 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-3f3a3094-dd9b-4f85-acac-6de032854974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:08:17.106364 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1347/2721] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:08:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:08:17.111064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08af0b70-2a8e-4d1d-bbb3-5e70483b788a None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:17.120952 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08af0b70-2a8e-4d1d-bbb3-5e70483b788a None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:17.125732 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ebfa92-08cd-4273-b2f5-aa444f6a5643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:17.126387 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ebfa92-08cd-4273-b2f5-aa444f6a5643 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:17.126671 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ebfa92-08cd-4273-b2f5-aa444f6a5643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:17.126948 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ebfa92-08cd-4273-b2f5-aa444f6a5643 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:17.127676 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ebfa92-08cd-4273-b2f5-aa444f6a5643 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 07:08:17.127983 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ebfa92-08cd-4273-b2f5-aa444f6a5643 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:17.129629 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ebfa92-08cd-4273-b2f5-aa444f6a5643 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:17.141339 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1375/2722] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 07:08:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 07:08:17.145567 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a3094-dd9b-4f85-acac-6de032854974 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:17.155010 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a3094-dd9b-4f85-acac-6de032854974 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:17.161380 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d9312-1a49-4fa8-a560-0e99f1c2578f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:17.162098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d9312-1a49-4fa8-a560-0e99f1c2578f admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:17.162385 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d9312-1a49-4fa8-a560-0e99f1c2578f admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:17.162665 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d9312-1a49-4fa8-a560-0e99f1c2578f admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:17.163363 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354d9312-1a49-4fa8-a560-0e99f1c2578f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:17.165286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354d9312-1a49-4fa8-a560-0e99f1c2578f admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:17.245220 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1348/2723] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 07:08:17 2022] POST /identity/v3/projects => generated 361 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 07:08:17.248684 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ebfa92-08cd-4273-b2f5-aa444f6a5643 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:17.258690 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ebfa92-08cd-4273-b2f5-aa444f6a5643 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:17.314867 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b939e682-4433-430d-ac9e-2bea1d5b599e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:17.315521 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939e682-4433-430d-ac9e-2bea1d5b599e admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:17.315810 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939e682-4433-430d-ac9e-2bea1d5b599e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:17.316148 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939e682-4433-430d-ac9e-2bea1d5b599e admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:17.316887 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b939e682-4433-430d-ac9e-2bea1d5b599e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:17.318802 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b939e682-4433-430d-ac9e-2bea1d5b599e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:17.431714 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1376/2724] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 07:08:17 2022] POST /identity/v3/users => generated 281 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 07:08:17.435866 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d9312-1a49-4fa8-a560-0e99f1c2578f admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:17.446398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d9312-1a49-4fa8-a560-0e99f1c2578f admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:17.451217 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ad64be-57d1-4b5c-a964-e86fe03ad373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:17.451875 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad64be-57d1-4b5c-a964-e86fe03ad373 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:17.452164 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad64be-57d1-4b5c-a964-e86fe03ad373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:17.452483 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad64be-57d1-4b5c-a964-e86fe03ad373 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:17.453764 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ad64be-57d1-4b5c-a964-e86fe03ad373 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:17.455350 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ad64be-57d1-4b5c-a964-e86fe03ad373 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:17.466508 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1349/2725] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:08:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:08:17.470989 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b939e682-4433-430d-ac9e-2bea1d5b599e admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:17.481230 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b939e682-4433-430d-ac9e-2bea1d5b599e admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:17.485958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77757b51-6b07-4c41-b660-53dbe89c3a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:17.486617 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77757b51-6b07-4c41-b660-53dbe89c3a3f admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:17.486946 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77757b51-6b07-4c41-b660-53dbe89c3a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:17.487216 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77757b51-6b07-4c41-b660-53dbe89c3a3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:17.487945 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77757b51-6b07-4c41-b660-53dbe89c3a3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:17.489573 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77757b51-6b07-4c41-b660-53dbe89c3a3f admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:17.499278 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1377/2726] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:08:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:08:17.503068 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ad64be-57d1-4b5c-a964-e86fe03ad373 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:17.512860 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ad64be-57d1-4b5c-a964-e86fe03ad373 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:17.518076 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7180ff-29a7-4ed0-b085-91554aff6318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:17.519010 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7180ff-29a7-4ed0-b085-91554aff6318 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:17.519238 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7180ff-29a7-4ed0-b085-91554aff6318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:17.519521 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7180ff-29a7-4ed0-b085-91554aff6318 admin admin] PATH_INFO: `/v3/projects/aca9421b72424581ab7cce81446b8fe8/users/49ebbbe3ce92424b97533b66207c64b3/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:17.534888 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7180ff-29a7-4ed0-b085-91554aff6318 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aca9421b72424581ab7cce81446b8fe8, user_id=49ebbbe3ce92424b97533b66207c64b3, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:17.537597 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7180ff-29a7-4ed0-b085-91554aff6318 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:17.597894 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1350/2727] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 07:08:17 2022] PUT /identity/v3/projects/aca9421b72424581ab7cce81446b8fe8/users/49ebbbe3ce92424b97533b66207c64b3/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:08:17.608233 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef479cb-1376-403c-8096-094230574d1b None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:17.608575 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef479cb-1376-403c-8096-094230574d1b None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:17.608890 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef479cb-1376-403c-8096-094230574d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:17.635703 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-9ef479cb-1376-403c-8096-094230574d1b None None] MFA Rules not processed for user `49ebbbe3ce92424b97533b66207c64b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:08:17.636964 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-9ef479cb-1376-403c-8096-094230574d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:08:17.668641 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:08:17.703958 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1378/2728] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:08:17 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:08:17.734390 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7180ff-29a7-4ed0-b085-91554aff6318 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:17.736235 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-6c7180ff-29a7-4ed0-b085-91554aff6318 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:08:17.751384 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:08:17.795923 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7180ff-29a7-4ed0-b085-91554aff6318 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:17.799882 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd0fcc3-1eb2-429b-b06b-21e088cbcc85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:17.800557 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0fcc3-1eb2-429b-b06b-21e088cbcc85 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:17.800845 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0fcc3-1eb2-429b-b06b-21e088cbcc85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:17.801125 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0fcc3-1eb2-429b-b06b-21e088cbcc85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:17.809667 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd0fcc3-1eb2-429b-b06b-21e088cbcc85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:17.811077 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd0fcc3-1eb2-429b-b06b-21e088cbcc85 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:17.825084 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1351/2729] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:08:17 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:08:17.839848 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef479cb-1376-403c-8096-094230574d1b None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:17.904041 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef479cb-1376-403c-8096-094230574d1b None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:17.909002 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d826f7-12dc-4426-9650-ca5d14d16e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:17.909872 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d826f7-12dc-4426-9650-ca5d14d16e80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:17.910159 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d826f7-12dc-4426-9650-ca5d14d16e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:17.910434 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d826f7-12dc-4426-9650-ca5d14d16e80 admin admin] PATH_INFO: `/v3/users/49ebbbe3ce92424b97533b66207c64b3` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:17.911616 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d826f7-12dc-4426-9650-ca5d14d16e80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49ebbbe3ce92424b97533b66207c64b3)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:17.913589 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d826f7-12dc-4426-9650-ca5d14d16e80 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:18.142888 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d3d826f7-12dc-4426-9650-ca5d14d16e80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49ebbbe3ce92424b97533b66207c64b3'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:18.150561 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d3d826f7-12dc-4426-9650-ca5d14d16e80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49ebbbe3ce92424b97533b66207c64b3'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:18.151262 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d3d826f7-12dc-4426-9650-ca5d14d16e80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49ebbbe3ce92424b97533b66207c64b3'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:18.151879 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d3d826f7-12dc-4426-9650-ca5d14d16e80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49ebbbe3ce92424b97533b66207c64b3'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:18.164388 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d3d826f7-12dc-4426-9650-ca5d14d16e80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49ebbbe3ce92424b97533b66207c64b3'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:18.288721 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1379/2730] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 07:08:17 2022] DELETE /identity/v3/users/49ebbbe3ce92424b97533b66207c64b3 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:08:18.293170 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef94a36-fcda-4d0a-9074-dc233a575d61 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:18.293472 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef94a36-fcda-4d0a-9074-dc233a575d61 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:18.293760 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef94a36-fcda-4d0a-9074-dc233a575d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:18.311743 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-7ef94a36-fcda-4d0a-9074-dc233a575d61 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:08:18.314427 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7ef94a36-fcda-4d0a-9074-dc233a575d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:08:18.372038 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1352/2731] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:08:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:08:18.381488 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d826f7-12dc-4426-9650-ca5d14d16e80 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:18.382975 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d3d826f7-12dc-4426-9650-ca5d14d16e80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:08:18.440865 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d826f7-12dc-4426-9650-ca5d14d16e80 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:18.444449 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e9878-a9c0-4a97-b36f-295b8b32fa99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:18.445106 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e9878-a9c0-4a97-b36f-295b8b32fa99 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:18.445395 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e9878-a9c0-4a97-b36f-295b8b32fa99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:18.445678 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e9878-a9c0-4a97-b36f-295b8b32fa99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:18.451543 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:08:18.455243 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03e9878-a9c0-4a97-b36f-295b8b32fa99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:18.456978 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03e9878-a9c0-4a97-b36f-295b8b32fa99 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:18.466588 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1380/2732] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:08:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:08:18.860909 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef94a36-fcda-4d0a-9074-dc233a575d61 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:18.862337 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7ef94a36-fcda-4d0a-9074-dc233a575d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:08:18.880383 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef94a36-fcda-4d0a-9074-dc233a575d61 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:18.885121 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a2d46-031a-4915-8ca9-b90abc52cf77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:18.885987 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a2d46-031a-4915-8ca9-b90abc52cf77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:18.886415 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a2d46-031a-4915-8ca9-b90abc52cf77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:18.886761 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a2d46-031a-4915-8ca9-b90abc52cf77 admin admin] PATH_INFO: `/v3/projects/aca9421b72424581ab7cce81446b8fe8` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:18.887920 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192a2d46-031a-4915-8ca9-b90abc52cf77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aca9421b72424581ab7cce81446b8fe8)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:18.892402 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192a2d46-031a-4915-8ca9-b90abc52cf77 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:18.984882 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-192a2d46-031a-4915-8ca9-b90abc52cf77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aca9421b72424581ab7cce81446b8fe8', 'request_id': 'req-192a2d46-031a-4915-8ca9-b90abc52cf77'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:18.990932 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-192a2d46-031a-4915-8ca9-b90abc52cf77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aca9421b72424581ab7cce81446b8fe8', 'request_id': 'req-192a2d46-031a-4915-8ca9-b90abc52cf77'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:19.060239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-192a2d46-031a-4915-8ca9-b90abc52cf77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aca9421b72424581ab7cce81446b8fe8', 'request_id': 'req-192a2d46-031a-4915-8ca9-b90abc52cf77'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:19.072562 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-192a2d46-031a-4915-8ca9-b90abc52cf77 admin admin] The token cache is being invalidate because project aca9421b72424581ab7cce81446b8fe8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 07:08:19.072858 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-192a2d46-031a-4915-8ca9-b90abc52cf77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:19.073509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-192a2d46-031a-4915-8ca9-b90abc52cf77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:08:19.075064 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1353/2733] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 07:08:18 2022] DELETE /identity/v3/projects/aca9421b72424581ab7cce81446b8fe8 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:08:19.081067 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41025ce4-1e81-40be-b141-81cc28f28824 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:19.081364 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41025ce4-1e81-40be-b141-81cc28f28824 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:19.081649 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41025ce4-1e81-40be-b141-81cc28f28824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:19.099385 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-41025ce4-1e81-40be-b141-81cc28f28824 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:08:19.100632 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-41025ce4-1e81-40be-b141-81cc28f28824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:08:19.160801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1381/2734] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:08:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:08:19.168199 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8334987-c92b-41af-923d-ee48c8a42345 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:19.168800 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8334987-c92b-41af-923d-ee48c8a42345 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:19.168943 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8334987-c92b-41af-923d-ee48c8a42345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:19.190056 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-d8334987-c92b-41af-923d-ee48c8a42345 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:08:19.191265 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-d8334987-c92b-41af-923d-ee48c8a42345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:08:19.199320 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1354/2735] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:08:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:08:19.203699 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41025ce4-1e81-40be-b141-81cc28f28824 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:19.214439 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41025ce4-1e81-40be-b141-81cc28f28824 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:19.219021 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05986425-9efa-437e-b6ff-34272d18d98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:19.219697 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05986425-9efa-437e-b6ff-34272d18d98c admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:19.220158 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05986425-9efa-437e-b6ff-34272d18d98c admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:19.220472 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05986425-9efa-437e-b6ff-34272d18d98c admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:19.221265 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05986425-9efa-437e-b6ff-34272d18d98c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 07:08:19.221580 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05986425-9efa-437e-b6ff-34272d18d98c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:19.223249 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05986425-9efa-437e-b6ff-34272d18d98c admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:19.234683 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1382/2736] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 07:08:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 07:08:19.238993 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8334987-c92b-41af-923d-ee48c8a42345 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:19.248163 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8334987-c92b-41af-923d-ee48c8a42345 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:19.253396 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46279507-c921-4edf-ae00-ca8fb2db86d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:19.254273 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46279507-c921-4edf-ae00-ca8fb2db86d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:19.254564 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46279507-c921-4edf-ae00-ca8fb2db86d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:19.254849 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46279507-c921-4edf-ae00-ca8fb2db86d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:19.255648 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46279507-c921-4edf-ae00-ca8fb2db86d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:19.257455 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46279507-c921-4edf-ae00-ca8fb2db86d5 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:19.340436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1355/2737] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 07:08:19 2022] POST /identity/v3/projects => generated 371 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 07:08:19.344565 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05986425-9efa-437e-b6ff-34272d18d98c admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:19.354849 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05986425-9efa-437e-b6ff-34272d18d98c admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:19.411603 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd1b6b-10a7-4906-872a-7bccb1625e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:19.412201 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd1b6b-10a7-4906-872a-7bccb1625e29 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:19.412526 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd1b6b-10a7-4906-872a-7bccb1625e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:19.412811 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd1b6b-10a7-4906-872a-7bccb1625e29 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:19.413503 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffd1b6b-10a7-4906-872a-7bccb1625e29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:19.415438 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffd1b6b-10a7-4906-872a-7bccb1625e29 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:19.524216 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1383/2738] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 07:08:19 2022] POST /identity/v3/users => generated 286 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 07:08:19.527481 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46279507-c921-4edf-ae00-ca8fb2db86d5 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:19.537993 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46279507-c921-4edf-ae00-ca8fb2db86d5 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:19.542641 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9877ce-b897-41ac-9c94-0a1377aab57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:19.543296 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9877ce-b897-41ac-9c94-0a1377aab57e admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:19.543588 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9877ce-b897-41ac-9c94-0a1377aab57e admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:19.543875 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9877ce-b897-41ac-9c94-0a1377aab57e admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:19.544716 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9877ce-b897-41ac-9c94-0a1377aab57e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:19.546316 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9877ce-b897-41ac-9c94-0a1377aab57e admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:19.552249 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:08:19.557716 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1356/2739] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:08:19.561576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd1b6b-10a7-4906-872a-7bccb1625e29 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:19.572387 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd1b6b-10a7-4906-872a-7bccb1625e29 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:19.577204 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e66ce19-1aaf-4b3d-bc47-fc70d49ca1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:19.577855 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66ce19-1aaf-4b3d-bc47-fc70d49ca1a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:19.578141 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66ce19-1aaf-4b3d-bc47-fc70d49ca1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:19.578422 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66ce19-1aaf-4b3d-bc47-fc70d49ca1a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:19.579287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e66ce19-1aaf-4b3d-bc47-fc70d49ca1a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:19.581212 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e66ce19-1aaf-4b3d-bc47-fc70d49ca1a4 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:19.591300 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1384/2740] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:08:19.595397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9877ce-b897-41ac-9c94-0a1377aab57e admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:19.605333 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9877ce-b897-41ac-9c94-0a1377aab57e admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:19.610978 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b25276-1185-467e-b2c7-9d89c9211004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:19.611850 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b25276-1185-467e-b2c7-9d89c9211004 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:19.612137 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b25276-1185-467e-b2c7-9d89c9211004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:19.612442 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b25276-1185-467e-b2c7-9d89c9211004 admin admin] PATH_INFO: `/v3/projects/233364fba2b8438a90f84f8e8db8df22/users/3c205b0c807e4abe8c6e91936da6d64d/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:19.628186 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b25276-1185-467e-b2c7-9d89c9211004 admin admin] RBAC: Authorizing `identity:create_grant(project_id=233364fba2b8438a90f84f8e8db8df22, user_id=3c205b0c807e4abe8c6e91936da6d64d, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:19.630871 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b25276-1185-467e-b2c7-9d89c9211004 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:19.727979 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1357/2741] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 07:08:19 2022] PUT /identity/v3/projects/233364fba2b8438a90f84f8e8db8df22/users/3c205b0c807e4abe8c6e91936da6d64d/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:08:19.738253 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003869f1-c2ed-4d49-a008-25ce8f959304 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:19.738570 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003869f1-c2ed-4d49-a008-25ce8f959304 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:19.738850 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003869f1-c2ed-4d49-a008-25ce8f959304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:19.764789 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-003869f1-c2ed-4d49-a008-25ce8f959304 None None] MFA Rules not processed for user `3c205b0c807e4abe8c6e91936da6d64d`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:08:19.765978 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-003869f1-c2ed-4d49-a008-25ce8f959304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:08:19.814008 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:08:19.832289 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1385/2742] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:08:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:08:19.843757 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b25276-1185-467e-b2c7-9d89c9211004 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:19.845668 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b6b25276-1185-467e-b2c7-9d89c9211004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:08:19.925220 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b25276-1185-467e-b2c7-9d89c9211004 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:19.928882 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9bc28-c069-4996-bd18-f0add3ed7830 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1193330cb5644765a7abb77738929ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:19.929529 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9bc28-c069-4996-bd18-f0add3ed7830 service glance] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:19.929820 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9bc28-c069-4996-bd18-f0add3ed7830 service glance] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:19.930103 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9bc28-c069-4996-bd18-f0add3ed7830 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:19.939282 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9bc28-c069-4996-bd18-f0add3ed7830 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:19.940673 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9bc28-c069-4996-bd18-f0add3ed7830 service glance] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:19.949414 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1358/2743] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:08:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 07:08:29.869412 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=5 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:08:29.945409 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=45,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:08:39.301056 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003869f1-c2ed-4d49-a008-25ce8f959304 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:39.302649 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-003869f1-c2ed-4d49-a008-25ce8f959304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:08:39.386955 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003869f1-c2ed-4d49-a008-25ce8f959304 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:39.390536 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1124bfd-4936-4b5c-8663-ed14e521b840 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:39.391773 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1124bfd-4936-4b5c-8663-ed14e521b840 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:39.392113 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1124bfd-4936-4b5c-8663-ed14e521b840 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:39.392432 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1124bfd-4936-4b5c-8663-ed14e521b840 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:39.394251 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c1124bfd-4936-4b5c-8663-ed14e521b840 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:08:39.406073 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1124bfd-4936-4b5c-8663-ed14e521b840 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:39.407776 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1124bfd-4936-4b5c-8663-ed14e521b840 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:39.416349 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1386/2744] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:08:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:08:43.403938 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9bc28-c069-4996-bd18-f0add3ed7830 service glance] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:43.405351 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-cbb9bc28-c069-4996-bd18-f0add3ed7830 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:08:43.463155 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9bc28-c069-4996-bd18-f0add3ed7830 service glance] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:43.466920 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a832d53b-dc6c-4ca2-947d-c07149992162 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:43.467570 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832d53b-dc6c-4ca2-947d-c07149992162 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:43.467862 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832d53b-dc6c-4ca2-947d-c07149992162 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:43.468141 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832d53b-dc6c-4ca2-947d-c07149992162 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:43.470709 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a832d53b-dc6c-4ca2-947d-c07149992162 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:08:43.530772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a832d53b-dc6c-4ca2-947d-c07149992162 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:43.533088 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a832d53b-dc6c-4ca2-947d-c07149992162 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:43.542142 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1359/2745] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:08:43 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:08:49.412624 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=35 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:08:53.537822 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=48 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:08:59.533474 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1124bfd-4936-4b5c-8663-ed14e521b840 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:08:59.542255 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1124bfd-4936-4b5c-8663-ed14e521b840 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:08:59.545858 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54576a2d-45dc-4810-9b76-37d71369e03b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:08:59.546496 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54576a2d-45dc-4810-9b76-37d71369e03b service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:08:59.546806 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54576a2d-45dc-4810-9b76-37d71369e03b service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:08:59.547085 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54576a2d-45dc-4810-9b76-37d71369e03b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:08:59.549203 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-54576a2d-45dc-4810-9b76-37d71369e03b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:08:59.618989 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54576a2d-45dc-4810-9b76-37d71369e03b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:08:59.620431 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54576a2d-45dc-4810-9b76-37d71369e03b service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:08:59.632009 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1387/2746] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:08:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:09:09.626467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=34 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:09:21.519107 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a832d53b-dc6c-4ca2-947d-c07149992162 service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:09:21.582126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a832d53b-dc6c-4ca2-947d-c07149992162 service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:09:21.586878 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b0e583-b9f8-475d-99c7-113b4255f87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:09:21.587746 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0e583-b9f8-475d-99c7-113b4255f87b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:09:21.588035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0e583-b9f8-475d-99c7-113b4255f87b admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:09:21.588434 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0e583-b9f8-475d-99c7-113b4255f87b admin admin] PATH_INFO: `/v3/users/3c205b0c807e4abe8c6e91936da6d64d` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:09:21.589568 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b0e583-b9f8-475d-99c7-113b4255f87b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c205b0c807e4abe8c6e91936da6d64d)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:09:21.591519 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b0e583-b9f8-475d-99c7-113b4255f87b admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:09:21.786182 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-10b0e583-b9f8-475d-99c7-113b4255f87b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c205b0c807e4abe8c6e91936da6d64d'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:09:21.793300 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-10b0e583-b9f8-475d-99c7-113b4255f87b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c205b0c807e4abe8c6e91936da6d64d'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:09:21.805650 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-10b0e583-b9f8-475d-99c7-113b4255f87b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c205b0c807e4abe8c6e91936da6d64d'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:09:21.806323 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-10b0e583-b9f8-475d-99c7-113b4255f87b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c205b0c807e4abe8c6e91936da6d64d'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:09:21.806932 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-10b0e583-b9f8-475d-99c7-113b4255f87b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c205b0c807e4abe8c6e91936da6d64d'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:09:21.875381 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1360/2747] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 07:09:21 2022] DELETE /identity/v3/users/3c205b0c807e4abe8c6e91936da6d64d => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:09:21.879977 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6b9c0-207d-4497-aee5-02f28d1833e5 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:09:21.880434 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6b9c0-207d-4497-aee5-02f28d1833e5 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:09:21.880579 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6b9c0-207d-4497-aee5-02f28d1833e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:09:21.898555 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-e6c6b9c0-207d-4497-aee5-02f28d1833e5 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:09:21.899794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e6c6b9c0-207d-4497-aee5-02f28d1833e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:09:21.958660 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1388/2748] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:09:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:09:21.968273 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b0e583-b9f8-475d-99c7-113b4255f87b admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:09:21.969681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-10b0e583-b9f8-475d-99c7-113b4255f87b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:09:22.025462 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b0e583-b9f8-475d-99c7-113b4255f87b admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:09:22.029084 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cce6e5-b072-4ce3-8196-0746e9feb295 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:09:22.029723 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cce6e5-b072-4ce3-8196-0746e9feb295 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:09:22.030009 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cce6e5-b072-4ce3-8196-0746e9feb295 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:09:22.030286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cce6e5-b072-4ce3-8196-0746e9feb295 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:09:22.039890 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cce6e5-b072-4ce3-8196-0746e9feb295 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:09:22.041317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cce6e5-b072-4ce3-8196-0746e9feb295 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:09:22.050525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1361/2749] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:09:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:09:22.456001 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6b9c0-207d-4497-aee5-02f28d1833e5 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:09:22.457478 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e6c6b9c0-207d-4497-aee5-02f28d1833e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:09:22.471601 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6b9c0-207d-4497-aee5-02f28d1833e5 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:09:22.478057 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51242b24-d511-433c-b544-1b308bbb4352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:09:22.478935 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51242b24-d511-433c-b544-1b308bbb4352 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:09:22.479225 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51242b24-d511-433c-b544-1b308bbb4352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:09:22.479625 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51242b24-d511-433c-b544-1b308bbb4352 admin admin] PATH_INFO: `/v3/projects/233364fba2b8438a90f84f8e8db8df22` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:09:22.481785 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51242b24-d511-433c-b544-1b308bbb4352 admin admin] RBAC: Authorizing `identity:delete_project(project_id=233364fba2b8438a90f84f8e8db8df22)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:09:22.486941 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51242b24-d511-433c-b544-1b308bbb4352 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:09:22.625976 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-51242b24-d511-433c-b544-1b308bbb4352 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '233364fba2b8438a90f84f8e8db8df22', 'request_id': 'req-51242b24-d511-433c-b544-1b308bbb4352'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:09:22.632819 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-51242b24-d511-433c-b544-1b308bbb4352 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '233364fba2b8438a90f84f8e8db8df22', 'request_id': 'req-51242b24-d511-433c-b544-1b308bbb4352'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:09:22.717514 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-51242b24-d511-433c-b544-1b308bbb4352 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '233364fba2b8438a90f84f8e8db8df22', 'request_id': 'req-51242b24-d511-433c-b544-1b308bbb4352'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:09:22.729245 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-51242b24-d511-433c-b544-1b308bbb4352 admin admin] The token cache is being invalidate because project 233364fba2b8438a90f84f8e8db8df22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 07:09:22.729479 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-51242b24-d511-433c-b544-1b308bbb4352 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:09:22.730111 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-51242b24-d511-433c-b544-1b308bbb4352 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:09:22.731666 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1389/2750] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 07:09:22 2022] DELETE /identity/v3/projects/233364fba2b8438a90f84f8e8db8df22 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:09:32.046356 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:09:32.725946 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:09:41.583016 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cce6e5-b072-4ce3-8196-0746e9feb295 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:09:41.584436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-d7cce6e5-b072-4ce3-8196-0746e9feb295 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:09:41.596198 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cce6e5-b072-4ce3-8196-0746e9feb295 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:09:41.645613 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee4709-89fd-4210-b865-d31e30b7b6e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:09:41.646259 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee4709-89fd-4210-b865-d31e30b7b6e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:09:41.646557 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee4709-89fd-4210-b865-d31e30b7b6e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:09:41.646839 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee4709-89fd-4210-b865-d31e30b7b6e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:09:41.648675 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-07ee4709-89fd-4210-b865-d31e30b7b6e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:09:41.711489 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ee4709-89fd-4210-b865-d31e30b7b6e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:09:41.713011 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ee4709-89fd-4210-b865-d31e30b7b6e6 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:09:41.722492 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1362/2751] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:09:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:09:42.232882 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51242b24-d511-433c-b544-1b308bbb4352 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:09:42.234324 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-51242b24-d511-433c-b544-1b308bbb4352 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:09:42.305330 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51242b24-d511-433c-b544-1b308bbb4352 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:09:42.311048 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3e1ba1-0e3a-4b54-94df-ec5ee33bea21 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3560534427544cdabfd0ba702a0a165', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:09:42.311697 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e1ba1-0e3a-4b54-94df-ec5ee33bea21 service placement] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:09:42.311985 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e1ba1-0e3a-4b54-94df-ec5ee33bea21 service placement] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:09:42.312266 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e1ba1-0e3a-4b54-94df-ec5ee33bea21 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:09:42.314061 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4a3e1ba1-0e3a-4b54-94df-ec5ee33bea21 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:09:42.326030 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3e1ba1-0e3a-4b54-94df-ec5ee33bea21 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:09:42.327607 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3e1ba1-0e3a-4b54-94df-ec5ee33bea21 service placement] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:09:42.337776 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1390/2752] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:09:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:09:42.379709 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee4709-89fd-4210-b865-d31e30b7b6e6 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:09:42.381127 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-07ee4709-89fd-4210-b865-d31e30b7b6e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:09:42.393716 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee4709-89fd-4210-b865-d31e30b7b6e6 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:09:42.398337 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c4069-1372-40db-88a6-8387539fab32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:09:42.399197 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c4069-1372-40db-88a6-8387539fab32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:09:42.399481 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c4069-1372-40db-88a6-8387539fab32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:09:42.399758 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c4069-1372-40db-88a6-8387539fab32 admin admin] PATH_INFO: `/v3/users/de31ee0904954d55b877e173ddc2c8be` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:09:42.400940 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c4069-1372-40db-88a6-8387539fab32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de31ee0904954d55b877e173ddc2c8be)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:09:42.403185 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c4069-1372-40db-88a6-8387539fab32 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:09:42.571758 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-465c4069-1372-40db-88a6-8387539fab32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de31ee0904954d55b877e173ddc2c8be'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:09:42.578860 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-465c4069-1372-40db-88a6-8387539fab32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de31ee0904954d55b877e173ddc2c8be'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:09:42.591530 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-465c4069-1372-40db-88a6-8387539fab32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de31ee0904954d55b877e173ddc2c8be'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:09:42.592216 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-465c4069-1372-40db-88a6-8387539fab32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de31ee0904954d55b877e173ddc2c8be'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:09:42.592867 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-465c4069-1372-40db-88a6-8387539fab32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de31ee0904954d55b877e173ddc2c8be'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:09:42.665820 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1363/2753] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 07:09:42 2022] DELETE /identity/v3/users/de31ee0904954d55b877e173ddc2c8be => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:09:42.901032 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3e1ba1-0e3a-4b54-94df-ec5ee33bea21 service placement] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:09:42.913209 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3e1ba1-0e3a-4b54-94df-ec5ee33bea21 service placement] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:09:42.969578 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d089c2f6-eda1-43b6-8fcd-28e8634980bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:09:42.970439 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089c2f6-eda1-43b6-8fcd-28e8634980bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:09:42.970722 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089c2f6-eda1-43b6-8fcd-28e8634980bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:09:42.970999 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089c2f6-eda1-43b6-8fcd-28e8634980bb admin admin] PATH_INFO: `/v3/projects/8a5093b9dbae43a3bf51e35f3825e8e4` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:09:42.972160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d089c2f6-eda1-43b6-8fcd-28e8634980bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a5093b9dbae43a3bf51e35f3825e8e4)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:09:42.974084 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d089c2f6-eda1-43b6-8fcd-28e8634980bb admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:09:43.069969 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d089c2f6-eda1-43b6-8fcd-28e8634980bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a5093b9dbae43a3bf51e35f3825e8e4', 'request_id': 'req-d089c2f6-eda1-43b6-8fcd-28e8634980bb'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:09:43.076655 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d089c2f6-eda1-43b6-8fcd-28e8634980bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a5093b9dbae43a3bf51e35f3825e8e4', 'request_id': 'req-d089c2f6-eda1-43b6-8fcd-28e8634980bb'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:09:43.151030 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d089c2f6-eda1-43b6-8fcd-28e8634980bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a5093b9dbae43a3bf51e35f3825e8e4', 'request_id': 'req-d089c2f6-eda1-43b6-8fcd-28e8634980bb'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:09:43.161488 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d089c2f6-eda1-43b6-8fcd-28e8634980bb admin admin] The token cache is being invalidate because project 8a5093b9dbae43a3bf51e35f3825e8e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 07:09:43.161772 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d089c2f6-eda1-43b6-8fcd-28e8634980bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:09:43.162419 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d089c2f6-eda1-43b6-8fcd-28e8634980bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:09:43.163944 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1391/2754] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 07:09:42 2022] DELETE /identity/v3/projects/8a5093b9dbae43a3bf51e35f3825e8e4 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:09:43.573464 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c4069-1372-40db-88a6-8387539fab32 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:09:43.575602 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-465c4069-1372-40db-88a6-8387539fab32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:09:43.602289 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:09:43.634114 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c4069-1372-40db-88a6-8387539fab32 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:09:43.682338 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b937b-4018-4e55-ac0d-a769961a33dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:09:43.683641 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b937b-4018-4e55-ac0d-a769961a33dc service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:09:43.684046 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b937b-4018-4e55-ac0d-a769961a33dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:09:43.684337 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b937b-4018-4e55-ac0d-a769961a33dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:09:43.686234 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-bf4b937b-4018-4e55-ac0d-a769961a33dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:09:43.768628 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4b937b-4018-4e55-ac0d-a769961a33dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:09:43.770102 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4b937b-4018-4e55-ac0d-a769961a33dc service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:09:43.788282 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1364/2755] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:09:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:09:53.158512 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:09:53.785631 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=71 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:10:48.229441 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d089c2f6-eda1-43b6-8fcd-28e8634980bb admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:10:48.230997 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d089c2f6-eda1-43b6-8fcd-28e8634980bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:10:48.314845 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d089c2f6-eda1-43b6-8fcd-28e8634980bb admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:10:48.320017 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2b239-1b99-49ec-a2c2-1c7a28665577 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3560534427544cdabfd0ba702a0a165', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:10:48.320703 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2b239-1b99-49ec-a2c2-1c7a28665577 service placement] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:10:48.320996 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2b239-1b99-49ec-a2c2-1c7a28665577 service placement] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:10:48.321391 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2b239-1b99-49ec-a2c2-1c7a28665577 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:10:48.323200 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-2eb2b239-1b99-49ec-a2c2-1c7a28665577 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:10:48.335811 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb2b239-1b99-49ec-a2c2-1c7a28665577 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:10:48.337432 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb2b239-1b99-49ec-a2c2-1c7a28665577 service placement] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:10:48.347440 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1392/2756] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:10:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:10:58.343137 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=37 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:12:43.696681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b937b-4018-4e55-ac0d-a769961a33dc service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:12:43.698128 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-bf4b937b-4018-4e55-ac0d-a769961a33dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:12:43.709648 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b937b-4018-4e55-ac0d-a769961a33dc service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:12:43.713227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099a2e88-e3e8-4bf1-8e54-d2133c9fa150 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:12:43.713853 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a2e88-e3e8-4bf1-8e54-d2133c9fa150 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:12:43.714133 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a2e88-e3e8-4bf1-8e54-d2133c9fa150 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:12:43.714406 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a2e88-e3e8-4bf1-8e54-d2133c9fa150 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:12:43.716825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-099a2e88-e3e8-4bf1-8e54-d2133c9fa150 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:12:43.782880 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099a2e88-e3e8-4bf1-8e54-d2133c9fa150 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:12:43.784306 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099a2e88-e3e8-4bf1-8e54-d2133c9fa150 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:12:43.802529 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1365/2757] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:12:43 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:12:53.799898 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=36 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:13:41.905303 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2b239-1b99-49ec-a2c2-1c7a28665577 service placement] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:13:41.906784 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-2eb2b239-1b99-49ec-a2c2-1c7a28665577 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:13:41.963733 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2b239-1b99-49ec-a2c2-1c7a28665577 service placement] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:13:41.967636 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064002b5-8a5a-4510-8d54-0d4040c051ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:13:41.968283 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064002b5-8a5a-4510-8d54-0d4040c051ce service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:13:41.968616 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064002b5-8a5a-4510-8d54-0d4040c051ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:13:41.968900 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064002b5-8a5a-4510-8d54-0d4040c051ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:13:41.970687 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-064002b5-8a5a-4510-8d54-0d4040c051ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:13:41.980076 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064002b5-8a5a-4510-8d54-0d4040c051ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:13:41.982082 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064002b5-8a5a-4510-8d54-0d4040c051ce service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:13:41.990181 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1393/2758] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:13:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:13:46.378157 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099a2e88-e3e8-4bf1-8e54-d2133c9fa150 service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:13:46.379655 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-099a2e88-e3e8-4bf1-8e54-d2133c9fa150 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:13:46.396578 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099a2e88-e3e8-4bf1-8e54-d2133c9fa150 service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:13:46.397493 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d1ec5-599c-4931-b20a-a2123113e8bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:13:46.398149 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d1ec5-599c-4931-b20a-a2123113e8bf service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:13:46.398438 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d1ec5-599c-4931-b20a-a2123113e8bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:13:46.398716 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d1ec5-599c-4931-b20a-a2123113e8bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:13:46.400540 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-c74d1ec5-599c-4931-b20a-a2123113e8bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:13:46.465057 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74d1ec5-599c-4931-b20a-a2123113e8bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:13:46.467221 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74d1ec5-599c-4931-b20a-a2123113e8bf service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:13:46.476964 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1366/2759] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:13:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 07:13:47.470037 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064002b5-8a5a-4510-8d54-0d4040c051ce service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:13:47.479528 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064002b5-8a5a-4510-8d54-0d4040c051ce service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:13:47.483953 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32246f47-343c-404e-91b4-d6f027ab6393 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:13:47.484657 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32246f47-343c-404e-91b4-d6f027ab6393 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:13:47.484947 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32246f47-343c-404e-91b4-d6f027ab6393 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:13:47.490872 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32246f47-343c-404e-91b4-d6f027ab6393 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:13:47.490872 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-32246f47-343c-404e-91b4-d6f027ab6393 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:13:47.568010 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32246f47-343c-404e-91b4-d6f027ab6393 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:13:47.569527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32246f47-343c-404e-91b4-d6f027ab6393 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:13:47.582965 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1394/2760] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:13:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:13:48.834577 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d1ec5-599c-4931-b20a-a2123113e8bf service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:13:48.836255 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-c74d1ec5-599c-4931-b20a-a2123113e8bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:13:48.861283 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d1ec5-599c-4931-b20a-a2123113e8bf service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:13:48.869181 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b816d317-b0b9-4293-993f-e57d8675f1b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:13:48.870149 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816d317-b0b9-4293-993f-e57d8675f1b7 service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:13:48.870602 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816d317-b0b9-4293-993f-e57d8675f1b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:13:48.871016 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816d317-b0b9-4293-993f-e57d8675f1b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:13:48.874087 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b816d317-b0b9-4293-993f-e57d8675f1b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:13:48.889574 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b816d317-b0b9-4293-993f-e57d8675f1b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:13:48.890830 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b816d317-b0b9-4293-993f-e57d8675f1b7 service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:13:48.905849 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1367/2761] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:13:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:13:50.561229 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32246f47-343c-404e-91b4-d6f027ab6393 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:13:50.573071 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32246f47-343c-404e-91b4-d6f027ab6393 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:13:50.579440 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a9ba3-54e9-4c82-92dd-9d30b00ce9bd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3560534427544cdabfd0ba702a0a165', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:13:50.580098 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a9ba3-54e9-4c82-92dd-9d30b00ce9bd service placement] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:13:50.580388 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a9ba3-54e9-4c82-92dd-9d30b00ce9bd service placement] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:13:50.580710 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a9ba3-54e9-4c82-92dd-9d30b00ce9bd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:13:50.582659 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-9e8a9ba3-54e9-4c82-92dd-9d30b00ce9bd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:13:50.595194 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a9ba3-54e9-4c82-92dd-9d30b00ce9bd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:13:50.596826 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a9ba3-54e9-4c82-92dd-9d30b00ce9bd service placement] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:13:50.607768 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1395/2762] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:13:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:13:54.602251 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b816d317-b0b9-4293-993f-e57d8675f1b7 service nova] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:13:54.603731 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b816d317-b0b9-4293-993f-e57d8675f1b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:13:54.619912 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b816d317-b0b9-4293-993f-e57d8675f1b7 service nova] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:13:54.626599 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b2e7c-f89d-420c-aed3-8bbff6e4c3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:13:54.627463 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b2e7c-f89d-420c-aed3-8bbff6e4c3e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:13:54.627748 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b2e7c-f89d-420c-aed3-8bbff6e4c3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:13:54.628026 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b2e7c-f89d-420c-aed3-8bbff6e4c3e1 admin admin] PATH_INFO: `/v3/users/621a7ea4f60545d5a5c30b2b7a370bf0` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:13:54.629277 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4b2e7c-f89d-420c-aed3-8bbff6e4c3e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=621a7ea4f60545d5a5c30b2b7a370bf0)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:13:54.631164 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4b2e7c-f89d-420c-aed3-8bbff6e4c3e1 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:13:54.838552 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-bc4b2e7c-f89d-420c-aed3-8bbff6e4c3e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '621a7ea4f60545d5a5c30b2b7a370bf0'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:13:54.846307 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-bc4b2e7c-f89d-420c-aed3-8bbff6e4c3e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '621a7ea4f60545d5a5c30b2b7a370bf0'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:13:54.858444 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-bc4b2e7c-f89d-420c-aed3-8bbff6e4c3e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '621a7ea4f60545d5a5c30b2b7a370bf0'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:13:54.859167 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-bc4b2e7c-f89d-420c-aed3-8bbff6e4c3e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '621a7ea4f60545d5a5c30b2b7a370bf0'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:13:54.859780 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-bc4b2e7c-f89d-420c-aed3-8bbff6e4c3e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '621a7ea4f60545d5a5c30b2b7a370bf0'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:13:54.983566 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1368/2763] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 07:13:54 2022] DELETE /identity/v3/users/621a7ea4f60545d5a5c30b2b7a370bf0 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:13:55.213860 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a9ba3-54e9-4c82-92dd-9d30b00ce9bd service placement] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:13:55.224986 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a9ba3-54e9-4c82-92dd-9d30b00ce9bd service placement] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:13:55.283527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ec631-233b-4a1d-af74-49bd21d546db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:13:55.284432 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ec631-233b-4a1d-af74-49bd21d546db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:13:55.284741 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ec631-233b-4a1d-af74-49bd21d546db admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:13:55.285024 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ec631-233b-4a1d-af74-49bd21d546db admin admin] PATH_INFO: `/v3/projects/fbaf0edf7bae44dfb37603b5648ecb64` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:13:55.286212 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698ec631-233b-4a1d-af74-49bd21d546db admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbaf0edf7bae44dfb37603b5648ecb64)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:13:55.288134 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698ec631-233b-4a1d-af74-49bd21d546db admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:13:55.305103 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:13:55.401482 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-698ec631-233b-4a1d-af74-49bd21d546db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbaf0edf7bae44dfb37603b5648ecb64', 'request_id': 'req-698ec631-233b-4a1d-af74-49bd21d546db'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:13:55.408480 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-698ec631-233b-4a1d-af74-49bd21d546db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbaf0edf7bae44dfb37603b5648ecb64', 'request_id': 'req-698ec631-233b-4a1d-af74-49bd21d546db'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:13:55.463609 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-698ec631-233b-4a1d-af74-49bd21d546db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbaf0edf7bae44dfb37603b5648ecb64', 'request_id': 'req-698ec631-233b-4a1d-af74-49bd21d546db'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:13:55.475021 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-698ec631-233b-4a1d-af74-49bd21d546db admin admin] The token cache is being invalidate because project fbaf0edf7bae44dfb37603b5648ecb64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 07:13:55.475311 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-698ec631-233b-4a1d-af74-49bd21d546db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:13:55.475950 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-698ec631-233b-4a1d-af74-49bd21d546db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:13:55.477751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1396/2764] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 07:13:55 2022] DELETE /identity/v3/projects/fbaf0edf7bae44dfb37603b5648ecb64 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:13:55.482118 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b2e7c-f89d-420c-aed3-8bbff6e4c3e1 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:13:55.483546 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-bc4b2e7c-f89d-420c-aed3-8bbff6e4c3e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:13:55.497610 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b2e7c-f89d-420c-aed3-8bbff6e4c3e1 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:13:55.555747 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:13:55.565629 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4add82b-f184-4903-828c-9318b842ec41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:13:55.567986 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4add82b-f184-4903-828c-9318b842ec41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:13:55.567986 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4add82b-f184-4903-828c-9318b842ec41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:13:55.567986 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4add82b-f184-4903-828c-9318b842ec41 admin admin] PATH_INFO: `/v3/users/2dc107028e2e4457b1e862669fb1d876` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:13:55.582650 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4add82b-f184-4903-828c-9318b842ec41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dc107028e2e4457b1e862669fb1d876)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:13:55.585261 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4add82b-f184-4903-828c-9318b842ec41 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:13:55.753934 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-a4add82b-f184-4903-828c-9318b842ec41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dc107028e2e4457b1e862669fb1d876'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:13:55.761233 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-a4add82b-f184-4903-828c-9318b842ec41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dc107028e2e4457b1e862669fb1d876'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:13:55.779441 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-a4add82b-f184-4903-828c-9318b842ec41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dc107028e2e4457b1e862669fb1d876'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:13:55.779441 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-a4add82b-f184-4903-828c-9318b842ec41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dc107028e2e4457b1e862669fb1d876'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:13:55.779441 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-a4add82b-f184-4903-828c-9318b842ec41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dc107028e2e4457b1e862669fb1d876'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:13:55.875646 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1369/2765] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 07:13:55 2022] DELETE /identity/v3/users/2dc107028e2e4457b1e862669fb1d876 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:13:56.119244 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ec631-233b-4a1d-af74-49bd21d546db admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:13:56.119244 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-698ec631-233b-4a1d-af74-49bd21d546db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:13:56.131858 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ec631-233b-4a1d-af74-49bd21d546db admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:13:56.189152 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50dbc7-ed51-4fea-a87a-f339eb7f49e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:13:56.190015 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50dbc7-ed51-4fea-a87a-f339eb7f49e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:13:56.190303 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50dbc7-ed51-4fea-a87a-f339eb7f49e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:13:56.190585 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50dbc7-ed51-4fea-a87a-f339eb7f49e7 admin admin] PATH_INFO: `/v3/projects/688a4570d2ed42869afcf8cb84f6af89` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:13:56.201569 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa50dbc7-ed51-4fea-a87a-f339eb7f49e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=688a4570d2ed42869afcf8cb84f6af89)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:13:56.203778 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa50dbc7-ed51-4fea-a87a-f339eb7f49e7 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:13:56.339337 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-aa50dbc7-ed51-4fea-a87a-f339eb7f49e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '688a4570d2ed42869afcf8cb84f6af89', 'request_id': 'req-aa50dbc7-ed51-4fea-a87a-f339eb7f49e7'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:13:56.346526 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-aa50dbc7-ed51-4fea-a87a-f339eb7f49e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '688a4570d2ed42869afcf8cb84f6af89', 'request_id': 'req-aa50dbc7-ed51-4fea-a87a-f339eb7f49e7'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:13:56.440343 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-aa50dbc7-ed51-4fea-a87a-f339eb7f49e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '688a4570d2ed42869afcf8cb84f6af89', 'request_id': 'req-aa50dbc7-ed51-4fea-a87a-f339eb7f49e7'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:13:56.451923 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-aa50dbc7-ed51-4fea-a87a-f339eb7f49e7 admin admin] The token cache is being invalidate because project 688a4570d2ed42869afcf8cb84f6af89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 07:13:56.452214 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-aa50dbc7-ed51-4fea-a87a-f339eb7f49e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:13:56.452867 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-aa50dbc7-ed51-4fea-a87a-f339eb7f49e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:13:56.454654 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1397/2766] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 07:13:56 2022] DELETE /identity/v3/projects/688a4570d2ed42869afcf8cb84f6af89 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:13:56.900756 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4add82b-f184-4903-828c-9318b842ec41 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:13:56.902158 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a4add82b-f184-4903-828c-9318b842ec41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:13:56.958931 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4add82b-f184-4903-828c-9318b842ec41 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:13:57.006501 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46134c-9d93-487e-b14a-56db225b3ba2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:13:57.007177 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46134c-9d93-487e-b14a-56db225b3ba2 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:13:57.007464 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46134c-9d93-487e-b14a-56db225b3ba2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:13:57.007744 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46134c-9d93-487e-b14a-56db225b3ba2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:13:57.009536 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-0c46134c-9d93-487e-b14a-56db225b3ba2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:13:57.039436 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:13:57.070689 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c46134c-9d93-487e-b14a-56db225b3ba2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:13:57.072114 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c46134c-9d93-487e-b14a-56db225b3ba2 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:13:57.082205 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1370/2767] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:13:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:14:00.646387 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50dbc7-ed51-4fea-a87a-f339eb7f49e7 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:14:00.647839 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-aa50dbc7-ed51-4fea-a87a-f339eb7f49e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:14:00.664916 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50dbc7-ed51-4fea-a87a-f339eb7f49e7 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:14:00.670003 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324c59b6-df4b-40a1-8269-e5ec1dd1fe8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:14:00.670884 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c59b6-df4b-40a1-8269-e5ec1dd1fe8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:14:00.674671 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c59b6-df4b-40a1-8269-e5ec1dd1fe8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:14:00.674671 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c59b6-df4b-40a1-8269-e5ec1dd1fe8e admin admin] PATH_INFO: `/v3/users/683c8a9abae1455bb2ec773b7608d03c` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:14:00.674671 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324c59b6-df4b-40a1-8269-e5ec1dd1fe8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=683c8a9abae1455bb2ec773b7608d03c)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:14:00.675308 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324c59b6-df4b-40a1-8269-e5ec1dd1fe8e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:14:00.697908 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:14:00.876723 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-324c59b6-df4b-40a1-8269-e5ec1dd1fe8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '683c8a9abae1455bb2ec773b7608d03c'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:14:00.883457 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-324c59b6-df4b-40a1-8269-e5ec1dd1fe8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '683c8a9abae1455bb2ec773b7608d03c'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:14:00.884107 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-324c59b6-df4b-40a1-8269-e5ec1dd1fe8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '683c8a9abae1455bb2ec773b7608d03c'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:14:00.884828 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-324c59b6-df4b-40a1-8269-e5ec1dd1fe8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '683c8a9abae1455bb2ec773b7608d03c'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:14:00.897131 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-324c59b6-df4b-40a1-8269-e5ec1dd1fe8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '683c8a9abae1455bb2ec773b7608d03c'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:14:00.977094 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1398/2768] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 07:14:00 2022] DELETE /identity/v3/users/683c8a9abae1455bb2ec773b7608d03c => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:14:01.232663 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46134c-9d93-487e-b14a-56db225b3ba2 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:14:01.242555 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46134c-9d93-487e-b14a-56db225b3ba2 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:14:01.302128 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d553a-c754-47bd-baa0-593f4a55bd06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:14:01.303051 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d553a-c754-47bd-baa0-593f4a55bd06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:14:01.303267 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d553a-c754-47bd-baa0-593f4a55bd06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:14:01.303546 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d553a-c754-47bd-baa0-593f4a55bd06 admin admin] PATH_INFO: `/v3/projects/2e45db85b4354b6ea087cd43addc110c` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:14:01.304773 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d553a-c754-47bd-baa0-593f4a55bd06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e45db85b4354b6ea087cd43addc110c)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:14:01.306708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d553a-c754-47bd-baa0-593f4a55bd06 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:14:01.414235 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-768d553a-c754-47bd-baa0-593f4a55bd06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e45db85b4354b6ea087cd43addc110c', 'request_id': 'req-768d553a-c754-47bd-baa0-593f4a55bd06'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:14:01.420674 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-768d553a-c754-47bd-baa0-593f4a55bd06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e45db85b4354b6ea087cd43addc110c', 'request_id': 'req-768d553a-c754-47bd-baa0-593f4a55bd06'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:14:01.503721 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-768d553a-c754-47bd-baa0-593f4a55bd06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e45db85b4354b6ea087cd43addc110c', 'request_id': 'req-768d553a-c754-47bd-baa0-593f4a55bd06'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:14:01.515394 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-768d553a-c754-47bd-baa0-593f4a55bd06 admin admin] The token cache is being invalidate because project 2e45db85b4354b6ea087cd43addc110c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 07:14:01.518384 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-768d553a-c754-47bd-baa0-593f4a55bd06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:14:01.518384 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-768d553a-c754-47bd-baa0-593f4a55bd06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:14:01.518384 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1371/2769] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 07:14:01 2022] DELETE /identity/v3/projects/2e45db85b4354b6ea087cd43addc110c => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:14:01.522965 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82414c3b-c72d-4710-850e-544e0d874532 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:14:01.523269 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82414c3b-c72d-4710-850e-544e0d874532 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:14:01.523549 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82414c3b-c72d-4710-850e-544e0d874532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:14:01.541734 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-82414c3b-c72d-4710-850e-544e0d874532 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:14:01.542981 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-82414c3b-c72d-4710-850e-544e0d874532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:14:01.603431 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1399/2770] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:14:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:14:01.611723 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d471ec-6862-444d-8777-762f746449cb None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:14:01.612182 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d471ec-6862-444d-8777-762f746449cb None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:14:01.612494 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d471ec-6862-444d-8777-762f746449cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:14:01.631060 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-49d471ec-6862-444d-8777-762f746449cb None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:14:01.632272 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-49d471ec-6862-444d-8777-762f746449cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:14:01.640144 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1372/2771] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:14:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:14:01.644276 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82414c3b-c72d-4710-850e-544e0d874532 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:14:01.653949 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82414c3b-c72d-4710-850e-544e0d874532 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:14:01.658612 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdadf6f-c84d-4bb4-8bbe-6a55fff74350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:14:01.659290 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdadf6f-c84d-4bb4-8bbe-6a55fff74350 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:14:01.659560 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdadf6f-c84d-4bb4-8bbe-6a55fff74350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:14:01.659843 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdadf6f-c84d-4bb4-8bbe-6a55fff74350 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:14:01.660608 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdadf6f-c84d-4bb4-8bbe-6a55fff74350 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 07:14:01.660922 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdadf6f-c84d-4bb4-8bbe-6a55fff74350 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:14:01.662565 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdadf6f-c84d-4bb4-8bbe-6a55fff74350 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:14:01.675467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1400/2772] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 07:14:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 07:14:01.679499 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d471ec-6862-444d-8777-762f746449cb None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:14:01.689708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d471ec-6862-444d-8777-762f746449cb None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:14:01.694428 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dedaca5-4bad-4e80-9761-6fa5b94d5c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:14:01.695076 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dedaca5-4bad-4e80-9761-6fa5b94d5c82 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:14:01.695363 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dedaca5-4bad-4e80-9761-6fa5b94d5c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:14:01.695641 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dedaca5-4bad-4e80-9761-6fa5b94d5c82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:14:01.696374 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dedaca5-4bad-4e80-9761-6fa5b94d5c82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:14:01.698330 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dedaca5-4bad-4e80-9761-6fa5b94d5c82 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:14:01.779535 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1373/2773] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 07:14:01 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 07:14:01.783679 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdadf6f-c84d-4bb4-8bbe-6a55fff74350 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:14:01.793673 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdadf6f-c84d-4bb4-8bbe-6a55fff74350 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:14:01.850515 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81925e-64f6-4c76-a805-7a98ce37dcbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:14:01.851179 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81925e-64f6-4c76-a805-7a98ce37dcbb admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:14:01.851467 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81925e-64f6-4c76-a805-7a98ce37dcbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:14:01.851747 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81925e-64f6-4c76-a805-7a98ce37dcbb admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:14:01.852440 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d81925e-64f6-4c76-a805-7a98ce37dcbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:14:01.854358 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d81925e-64f6-4c76-a805-7a98ce37dcbb admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:14:01.926432 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1401/2774] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 07:14:01 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 07:14:01.930611 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dedaca5-4bad-4e80-9761-6fa5b94d5c82 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:14:01.941525 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dedaca5-4bad-4e80-9761-6fa5b94d5c82 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:14:01.946514 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c04c5-56c6-45ba-8194-16448d436447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:14:01.947188 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c04c5-56c6-45ba-8194-16448d436447 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:14:01.947472 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c04c5-56c6-45ba-8194-16448d436447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:14:01.947747 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c04c5-56c6-45ba-8194-16448d436447 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:14:01.948549 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c04c5-56c6-45ba-8194-16448d436447 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:14:01.950128 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c04c5-56c6-45ba-8194-16448d436447 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:14:01.961776 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1374/2775] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:14:01.965868 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81925e-64f6-4c76-a805-7a98ce37dcbb admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:14:01.975868 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81925e-64f6-4c76-a805-7a98ce37dcbb admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:14:01.980437 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b665f9e-4902-475f-a49c-67278c67fa5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:14:01.981093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b665f9e-4902-475f-a49c-67278c67fa5b admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:14:01.981379 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b665f9e-4902-475f-a49c-67278c67fa5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:14:01.981650 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b665f9e-4902-475f-a49c-67278c67fa5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:14:01.982451 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b665f9e-4902-475f-a49c-67278c67fa5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:14:01.984030 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b665f9e-4902-475f-a49c-67278c67fa5b admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:14:01.993808 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1402/2776] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:14:01.998255 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c04c5-56c6-45ba-8194-16448d436447 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:14:02.008074 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c04c5-56c6-45ba-8194-16448d436447 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:14:02.012663 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f22d5-e389-4f21-b1cd-f2dfa4862646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:14:02.014245 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f22d5-e389-4f21-b1cd-f2dfa4862646 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:14:02.014517 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f22d5-e389-4f21-b1cd-f2dfa4862646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:14:02.014813 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f22d5-e389-4f21-b1cd-f2dfa4862646 admin admin] PATH_INFO: `/v3/projects/c6c31a4ef7c3413aa6f7d2b9f08e4f63/users/fed79b23091040daa06000754399eb84/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:14:02.029051 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22f22d5-e389-4f21-b1cd-f2dfa4862646 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6c31a4ef7c3413aa6f7d2b9f08e4f63, user_id=fed79b23091040daa06000754399eb84, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:14:02.031743 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22f22d5-e389-4f21-b1cd-f2dfa4862646 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:14:02.033923 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:14:02.081532 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1375/2777] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 07:14:01 2022] PUT /identity/v3/projects/c6c31a4ef7c3413aa6f7d2b9f08e4f63/users/fed79b23091040daa06000754399eb84/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 07:14:02.086801 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef5eb63-9cb8-4db9-8001-496c33a47702 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:14:02.087135 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef5eb63-9cb8-4db9-8001-496c33a47702 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:14:02.087422 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef5eb63-9cb8-4db9-8001-496c33a47702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:14:02.094335 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:14:02.105481 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-6ef5eb63-9cb8-4db9-8001-496c33a47702 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:14:02.107227 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-6ef5eb63-9cb8-4db9-8001-496c33a47702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:14:02.165386 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1403/2778] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:14:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:14:02.175739 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f22d5-e389-4f21-b1cd-f2dfa4862646 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:14:02.177509 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-c22f22d5-e389-4f21-b1cd-f2dfa4862646 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:14:02.233882 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f22d5-e389-4f21-b1cd-f2dfa4862646 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:14:02.237535 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38565d57-7999-4903-81da-6b4316b0f629 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:14:02.238179 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38565d57-7999-4903-81da-6b4316b0f629 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:14:02.238468 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38565d57-7999-4903-81da-6b4316b0f629 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:14:02.238754 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38565d57-7999-4903-81da-6b4316b0f629 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:14:02.248546 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38565d57-7999-4903-81da-6b4316b0f629 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:14:02.249986 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38565d57-7999-4903-81da-6b4316b0f629 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:14:02.259584 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1376/2779] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:14:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:14:07.226020 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43515fa-35d0-491f-adef-c33b5bbf6dea None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:14:07.226474 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43515fa-35d0-491f-adef-c33b5bbf6dea None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:14:07.226626 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43515fa-35d0-491f-adef-c33b5bbf6dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:14:07.259207 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-c43515fa-35d0-491f-adef-c33b5bbf6dea None None] MFA Rules not processed for user `fed79b23091040daa06000754399eb84`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:14:07.263865 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c43515fa-35d0-491f-adef-c33b5bbf6dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:14:07.342234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1404/2780] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:14:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:14:07.358624 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38565d57-7999-4903-81da-6b4316b0f629 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:14:07.368319 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-38565d57-7999-4903-81da-6b4316b0f629 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:14:07.450405 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38565d57-7999-4903-81da-6b4316b0f629 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:14:07.458407 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a559795-ae7b-4374-a350-fec0817dbeb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:14:07.459246 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a559795-ae7b-4374-a350-fec0817dbeb9 service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:14:07.460038 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a559795-ae7b-4374-a350-fec0817dbeb9 service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:14:07.460477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a559795-ae7b-4374-a350-fec0817dbeb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:14:07.471898 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a559795-ae7b-4374-a350-fec0817dbeb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:14:07.473844 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a559795-ae7b-4374-a350-fec0817dbeb9 service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:14:07.485213 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1377/2781] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:14:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:14:09.717526 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695486d5-ab14-4c7d-965c-1c23033ab429 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:14:09.718022 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695486d5-ab14-4c7d-965c-1c23033ab429 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:14:09.718113 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695486d5-ab14-4c7d-965c-1c23033ab429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:14:09.737827 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-695486d5-ab14-4c7d-965c-1c23033ab429 None None] MFA Rules not processed for user `fed79b23091040daa06000754399eb84`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:14:09.739351 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-695486d5-ab14-4c7d-965c-1c23033ab429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:14:09.747740 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1405/2782] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:14:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:14:09.788267 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a559795-ae7b-4374-a350-fec0817dbeb9 service nova] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:14:09.789864 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-1a559795-ae7b-4374-a350-fec0817dbeb9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:14:09.861505 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a559795-ae7b-4374-a350-fec0817dbeb9 service nova] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:14:09.865536 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536f740-80fa-4125-9066-f7a707567558 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1193330cb5644765a7abb77738929ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:14:09.866197 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536f740-80fa-4125-9066-f7a707567558 service glance] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:14:09.866476 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536f740-80fa-4125-9066-f7a707567558 service glance] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:14:09.866757 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536f740-80fa-4125-9066-f7a707567558 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:14:09.869131 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4536f740-80fa-4125-9066-f7a707567558 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:14:09.885690 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536f740-80fa-4125-9066-f7a707567558 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:14:09.887202 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536f740-80fa-4125-9066-f7a707567558 service glance] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:14:09.902227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1378/2783] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:14:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:14:10.218155 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695486d5-ab14-4c7d-965c-1c23033ab429 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:14:10.223142 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-695486d5-ab14-4c7d-965c-1c23033ab429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:14:10.237580 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695486d5-ab14-4c7d-965c-1c23033ab429 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:14:10.241394 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75010ccf-ad01-462b-a1ac-6a43f3db5cf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:14:10.242064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75010ccf-ad01-462b-a1ac-6a43f3db5cf8 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:14:10.242357 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75010ccf-ad01-462b-a1ac-6a43f3db5cf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:14:10.242641 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75010ccf-ad01-462b-a1ac-6a43f3db5cf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:14:10.244528 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-75010ccf-ad01-462b-a1ac-6a43f3db5cf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:14:10.256033 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75010ccf-ad01-462b-a1ac-6a43f3db5cf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:14:10.257623 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75010ccf-ad01-462b-a1ac-6a43f3db5cf8 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:14:10.268606 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1406/2784] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:14:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:14:10.992647 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536f740-80fa-4125-9066-f7a707567558 service glance] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:14:10.994713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4536f740-80fa-4125-9066-f7a707567558 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:14:11.025644 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:14:11.074028 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536f740-80fa-4125-9066-f7a707567558 service glance] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:14:11.079785 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee13ee1-ea24-41a7-bf82-bae461d7c1a3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3560534427544cdabfd0ba702a0a165', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:14:11.080469 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee13ee1-ea24-41a7-bf82-bae461d7c1a3 service placement] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:14:11.085432 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee13ee1-ea24-41a7-bf82-bae461d7c1a3 service placement] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:14:11.085432 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee13ee1-ea24-41a7-bf82-bae461d7c1a3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:14:11.085432 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7ee13ee1-ea24-41a7-bf82-bae461d7c1a3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:14:11.095188 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee13ee1-ea24-41a7-bf82-bae461d7c1a3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:14:11.096789 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee13ee1-ea24-41a7-bf82-bae461d7c1a3 service placement] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:14:11.107546 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1379/2785] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:14:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:14:19.761211 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75010ccf-ad01-462b-a1ac-6a43f3db5cf8 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:14:19.762710 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-75010ccf-ad01-462b-a1ac-6a43f3db5cf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:14:19.779979 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75010ccf-ad01-462b-a1ac-6a43f3db5cf8 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:14:19.785520 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91649331-2e60-493d-8097-e5b766a869af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:14:19.786171 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91649331-2e60-493d-8097-e5b766a869af service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:14:19.786462 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91649331-2e60-493d-8097-e5b766a869af service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:14:19.786744 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91649331-2e60-493d-8097-e5b766a869af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:14:19.788567 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-91649331-2e60-493d-8097-e5b766a869af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:14:19.801644 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91649331-2e60-493d-8097-e5b766a869af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:14:19.803225 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91649331-2e60-493d-8097-e5b766a869af service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:14:19.813972 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1407/2786] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:14:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:14:21.103223 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=21 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:14:29.809324 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=72 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:15:53.274990 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35947e-8966-474c-9232-f4dd316ed812 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:15:53.275787 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35947e-8966-474c-9232-f4dd316ed812 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:15:53.275787 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35947e-8966-474c-9232-f4dd316ed812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:15:53.293231 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-de35947e-8966-474c-9232-f4dd316ed812 None None] MFA Rules not processed for user `fed79b23091040daa06000754399eb84`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:15:53.294354 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-de35947e-8966-474c-9232-f4dd316ed812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:15:53.301860 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1380/2787] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:15:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:15:53.613983 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87207f-8bd4-4c4f-bc0f-e96835ed6caf None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:15:53.614505 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87207f-8bd4-4c4f-bc0f-e96835ed6caf None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:15:53.614612 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87207f-8bd4-4c4f-bc0f-e96835ed6caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:15:53.661918 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-bf87207f-8bd4-4c4f-bc0f-e96835ed6caf None None] MFA Rules not processed for user `9824a96e2bcd4a16bfc6fe6bf4ea1ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:15:53.663129 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-bf87207f-8bd4-4c4f-bc0f-e96835ed6caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:15:53.679984 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1408/2788] 10.222.0.40 () {58 vars in 1062 bytes} [Mon Jul 18 07:15:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:15:53.686197 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35947e-8966-474c-9232-f4dd316ed812 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:15:53.695871 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35947e-8966-474c-9232-f4dd316ed812 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:15:53.699440 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2ae8f-f4a1-430f-9129-4c59598833bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:15:53.700075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2ae8f-f4a1-430f-9129-4c59598833bd service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:15:53.700363 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2ae8f-f4a1-430f-9129-4c59598833bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:15:53.700682 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2ae8f-f4a1-430f-9129-4c59598833bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:15:53.702694 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b3c2ae8f-f4a1-430f-9129-4c59598833bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:15:53.712686 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c2ae8f-f4a1-430f-9129-4c59598833bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:15:53.714095 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c2ae8f-f4a1-430f-9129-4c59598833bd service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:15:53.722936 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1381/2789] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:15:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:15:54.191099 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf87207f-8bd4-4c4f-bc0f-e96835ed6caf None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:15:54.216124 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf87207f-8bd4-4c4f-bc0f-e96835ed6caf None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:15:54.221276 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df75390-b10c-4ec2-b9b3-c8cb0c7fcf46 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3560534427544cdabfd0ba702a0a165', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:15:54.221922 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df75390-b10c-4ec2-b9b3-c8cb0c7fcf46 service placement] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:15:54.222243 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df75390-b10c-4ec2-b9b3-c8cb0c7fcf46 service placement] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:15:54.222565 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df75390-b10c-4ec2-b9b3-c8cb0c7fcf46 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:15:54.224553 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-2df75390-b10c-4ec2-b9b3-c8cb0c7fcf46 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:15:54.239955 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df75390-b10c-4ec2-b9b3-c8cb0c7fcf46 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:15:54.241613 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df75390-b10c-4ec2-b9b3-c8cb0c7fcf46 service placement] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:15:54.252671 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1409/2790] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:15:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:16:03.719175 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=9 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:16:04.248397 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=27 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:17:40.251046 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c153c1-8ebe-49ef-8817-d89901fe47ae None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:17:40.251565 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c153c1-8ebe-49ef-8817-d89901fe47ae None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:17:40.251663 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c153c1-8ebe-49ef-8817-d89901fe47ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:17:40.270224 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-40c153c1-8ebe-49ef-8817-d89901fe47ae None None] MFA Rules not processed for user `fed79b23091040daa06000754399eb84`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:17:40.271441 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-40c153c1-8ebe-49ef-8817-d89901fe47ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:17:40.278602 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1382/2791] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:17:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:17:40.963200 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379fbc7a-3ec2-46ef-99a9-fbb1e6e696e7 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:17:40.963686 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379fbc7a-3ec2-46ef-99a9-fbb1e6e696e7 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:17:40.963765 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379fbc7a-3ec2-46ef-99a9-fbb1e6e696e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:17:41.002181 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-379fbc7a-3ec2-46ef-99a9-fbb1e6e696e7 None None] MFA Rules not processed for user `a3560534427544cdabfd0ba702a0a165`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:17:41.003347 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-379fbc7a-3ec2-46ef-99a9-fbb1e6e696e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:17:41.012666 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1410/2792] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Jul 18 07:17:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:17:41.023980 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c153c1-8ebe-49ef-8817-d89901fe47ae None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:17:41.025361 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-40c153c1-8ebe-49ef-8817-d89901fe47ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:17:41.040157 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c153c1-8ebe-49ef-8817-d89901fe47ae None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:17:41.045073 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917396c-6f63-4f2b-be1f-493e93c8359a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3560534427544cdabfd0ba702a0a165', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:17:41.045706 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917396c-6f63-4f2b-be1f-493e93c8359a service placement] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:17:41.046506 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917396c-6f63-4f2b-be1f-493e93c8359a service placement] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:17:41.046862 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917396c-6f63-4f2b-be1f-493e93c8359a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:17:41.055191 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7917396c-6f63-4f2b-be1f-493e93c8359a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:17:41.056782 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7917396c-6f63-4f2b-be1f-493e93c8359a service placement] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:17:41.067129 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1383/2793] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:17:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:17:50.998976 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=9 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:17:51.063113 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=9 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:18:47.920397 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379fbc7a-3ec2-46ef-99a9-fbb1e6e696e7 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:18:47.948970 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379fbc7a-3ec2-46ef-99a9-fbb1e6e696e7 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:18:47.952563 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab9d71c-3b01-4a16-b32a-f4b8a29e542a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:18:47.953200 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab9d71c-3b01-4a16-b32a-f4b8a29e542a service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:18:47.953481 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab9d71c-3b01-4a16-b32a-f4b8a29e542a service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:18:47.953754 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab9d71c-3b01-4a16-b32a-f4b8a29e542a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:18:47.955495 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-0ab9d71c-3b01-4a16-b32a-f4b8a29e542a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:18:47.986493 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab9d71c-3b01-4a16-b32a-f4b8a29e542a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:18:47.987920 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab9d71c-3b01-4a16-b32a-f4b8a29e542a service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:18:47.996722 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1411/2794] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:18:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:18:57.993138 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=24 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:19:01.136479 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917396c-6f63-4f2b-be1f-493e93c8359a service placement] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:19:01.137934 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7917396c-6f63-4f2b-be1f-493e93c8359a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:19:01.226204 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917396c-6f63-4f2b-be1f-493e93c8359a service placement] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:19:01.229838 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db244780-d2e4-448a-88ba-98444f5dbc11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:19:01.230476 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db244780-d2e4-448a-88ba-98444f5dbc11 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:19:01.230839 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db244780-d2e4-448a-88ba-98444f5dbc11 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:19:01.231121 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db244780-d2e4-448a-88ba-98444f5dbc11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:19:01.232907 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-db244780-d2e4-448a-88ba-98444f5dbc11 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:19:01.286133 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db244780-d2e4-448a-88ba-98444f5dbc11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:19:01.288196 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db244780-d2e4-448a-88ba-98444f5dbc11 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:19:01.310531 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1384/2795] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:19:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:19:07.290834 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce55ea-ab88-423e-87d6-1f1bf9fbe8a4 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:19:07.291316 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce55ea-ab88-423e-87d6-1f1bf9fbe8a4 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:19:07.291416 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce55ea-ab88-423e-87d6-1f1bf9fbe8a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:19:07.337071 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-c7ce55ea-ab88-423e-87d6-1f1bf9fbe8a4 None None] MFA Rules not processed for user `5af3a4a1bc1f4c4e9653d0a790f12606`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:19:07.338279 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c7ce55ea-ab88-423e-87d6-1f1bf9fbe8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:19:07.344830 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1412/2796] 10.222.0.40 () {58 vars in 1061 bytes} [Mon Jul 18 07:19:07 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:19:07.351331 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db244780-d2e4-448a-88ba-98444f5dbc11 service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:19:07.359765 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db244780-d2e4-448a-88ba-98444f5dbc11 service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:19:07.363374 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b7368-4514-4e3c-8157-520696d74f65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:19:07.364010 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b7368-4514-4e3c-8157-520696d74f65 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:19:07.365086 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b7368-4514-4e3c-8157-520696d74f65 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:19:07.365374 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b7368-4514-4e3c-8157-520696d74f65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:19:07.373677 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74b7368-4514-4e3c-8157-520696d74f65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:19:07.375087 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74b7368-4514-4e3c-8157-520696d74f65 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:19:07.383863 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1385/2797] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:19:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:19:12.595098 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce55ea-ab88-423e-87d6-1f1bf9fbe8a4 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:19:12.604292 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce55ea-ab88-423e-87d6-1f1bf9fbe8a4 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:19:12.607846 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886d7458-e68b-41cf-b2c2-e7ae1b665689 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:19:12.608529 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d7458-e68b-41cf-b2c2-e7ae1b665689 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:19:12.608823 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d7458-e68b-41cf-b2c2-e7ae1b665689 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:19:12.609109 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d7458-e68b-41cf-b2c2-e7ae1b665689 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:19:12.619388 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886d7458-e68b-41cf-b2c2-e7ae1b665689 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:19:12.620857 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886d7458-e68b-41cf-b2c2-e7ae1b665689 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:19:12.632332 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1413/2798] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:19:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:19:15.569920 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b7368-4514-4e3c-8157-520696d74f65 service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:19:15.583633 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b7368-4514-4e3c-8157-520696d74f65 service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:19:15.590391 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df4235-1efa-4628-8e33-1ce283fa420e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:19:15.591045 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df4235-1efa-4628-8e33-1ce283fa420e service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:19:15.591334 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df4235-1efa-4628-8e33-1ce283fa420e service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:19:15.591624 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df4235-1efa-4628-8e33-1ce283fa420e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:19:15.593514 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-83df4235-1efa-4628-8e33-1ce283fa420e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:19:15.608134 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df4235-1efa-4628-8e33-1ce283fa420e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:19:15.609773 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df4235-1efa-4628-8e33-1ce283fa420e service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:19:15.634197 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1386/2799] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:19:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:19:20.342031 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac654b-719c-4a82-812a-e06a66872169 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:19:20.346948 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac654b-719c-4a82-812a-e06a66872169 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:19:20.346948 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac654b-719c-4a82-812a-e06a66872169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:19:20.360550 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-dcac654b-719c-4a82-812a-e06a66872169 None None] MFA Rules not processed for user `fed79b23091040daa06000754399eb84`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:19:20.361755 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-dcac654b-719c-4a82-812a-e06a66872169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:19:20.368780 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1414/2800] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:19:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:19:20.666927 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df4235-1efa-4628-8e33-1ce283fa420e service nova] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:19:20.676158 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df4235-1efa-4628-8e33-1ce283fa420e service nova] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:19:20.680282 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de16a5-50ed-4ea6-8869-252ccf9b4ccd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:19:20.680983 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de16a5-50ed-4ea6-8869-252ccf9b4ccd service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:19:20.681266 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de16a5-50ed-4ea6-8869-252ccf9b4ccd service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:19:20.681541 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de16a5-50ed-4ea6-8869-252ccf9b4ccd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:19:20.689330 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26de16a5-50ed-4ea6-8869-252ccf9b4ccd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:19:20.690746 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26de16a5-50ed-4ea6-8869-252ccf9b4ccd service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:19:20.699777 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1387/2801] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:19:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:19:21.090963 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76cc8b5-1b56-4e73-a1f3-613074cce9e1 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:19:21.091335 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76cc8b5-1b56-4e73-a1f3-613074cce9e1 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:19:21.091686 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76cc8b5-1b56-4e73-a1f3-613074cce9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:19:21.110382 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-d76cc8b5-1b56-4e73-a1f3-613074cce9e1 None None] MFA Rules not processed for user `a3560534427544cdabfd0ba702a0a165`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:19:21.111697 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d76cc8b5-1b56-4e73-a1f3-613074cce9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:19:21.121193 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1415/2802] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Jul 18 07:19:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 07:19:21.133379 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de16a5-50ed-4ea6-8869-252ccf9b4ccd service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:19:21.145549 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de16a5-50ed-4ea6-8869-252ccf9b4ccd service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:19:21.151687 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31937d05-d585-4896-b86b-5eba009638a2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3560534427544cdabfd0ba702a0a165', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:19:21.152334 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31937d05-d585-4896-b86b-5eba009638a2 service placement] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:19:21.152779 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31937d05-d585-4896-b86b-5eba009638a2 service placement] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:19:21.153057 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31937d05-d585-4896-b86b-5eba009638a2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:19:21.162466 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31937d05-d585-4896-b86b-5eba009638a2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:19:21.164042 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31937d05-d585-4896-b86b-5eba009638a2 service placement] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:19:21.174035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1388/2803] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:19:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:19:31.105121 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=25 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:19:31.169817 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=85 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:19:31.414619 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76cc8b5-1b56-4e73-a1f3-613074cce9e1 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:19:31.425066 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76cc8b5-1b56-4e73-a1f3-613074cce9e1 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:19:31.430616 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc3de0-0bb3-4323-a878-9fe374c3c61f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:19:31.431251 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc3de0-0bb3-4323-a878-9fe374c3c61f service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:19:31.431536 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc3de0-0bb3-4323-a878-9fe374c3c61f service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:19:31.431814 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc3de0-0bb3-4323-a878-9fe374c3c61f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:19:31.440829 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dc3de0-0bb3-4323-a878-9fe374c3c61f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:19:31.442398 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dc3de0-0bb3-4323-a878-9fe374c3c61f service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:19:31.454026 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1416/2804] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:19:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 07:19:41.449478 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=6 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:20:59.202046 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31937d05-d585-4896-b86b-5eba009638a2 service placement] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:20:59.212443 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31937d05-d585-4896-b86b-5eba009638a2 service placement] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:20:59.218007 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe1c36d-3ce9-4f96-9dfb-04e550cae19f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3560534427544cdabfd0ba702a0a165', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:20:59.218635 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1c36d-3ce9-4f96-9dfb-04e550cae19f service placement] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:20:59.218926 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1c36d-3ce9-4f96-9dfb-04e550cae19f service placement] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:20:59.219203 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1c36d-3ce9-4f96-9dfb-04e550cae19f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:20:59.227713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe1c36d-3ce9-4f96-9dfb-04e550cae19f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:20:59.229302 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe1c36d-3ce9-4f96-9dfb-04e550cae19f service placement] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:20:59.239704 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1389/2805] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:20:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:21:09.235521 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=6 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:21:13.565811 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc3de0-0bb3-4323-a878-9fe374c3c61f service nova] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:13.575413 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc3de0-0bb3-4323-a878-9fe374c3c61f service nova] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:13.579092 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808620a3-3f79-460f-8bcc-503984635d8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:13.579729 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808620a3-3f79-460f-8bcc-503984635d8a service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:13.580016 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808620a3-3f79-460f-8bcc-503984635d8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:13.580296 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808620a3-3f79-460f-8bcc-503984635d8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:13.589245 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808620a3-3f79-460f-8bcc-503984635d8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:13.590670 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808620a3-3f79-460f-8bcc-503984635d8a service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:13.618889 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1417/2806] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:21:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:21:17.131593 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe1c36d-3ce9-4f96-9dfb-04e550cae19f service placement] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:17.142590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe1c36d-3ce9-4f96-9dfb-04e550cae19f service placement] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:17.147696 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d2779-4e8b-451c-95c7-4d20df7229df service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3560534427544cdabfd0ba702a0a165', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:17.148330 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d2779-4e8b-451c-95c7-4d20df7229df service placement] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:17.148648 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d2779-4e8b-451c-95c7-4d20df7229df service placement] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:17.148927 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d2779-4e8b-451c-95c7-4d20df7229df service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:17.151030 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e09d2779-4e8b-451c-95c7-4d20df7229df service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:17.177517 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09d2779-4e8b-451c-95c7-4d20df7229df service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:17.179167 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09d2779-4e8b-451c-95c7-4d20df7229df service placement] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:17.189999 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1390/2807] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:21:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:21:17.269080 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808620a3-3f79-460f-8bcc-503984635d8a service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:17.279542 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808620a3-3f79-460f-8bcc-503984635d8a service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:17.284502 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c08348-8dcc-4b77-93ba-73e0bb9f5886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:17.285075 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c08348-8dcc-4b77-93ba-73e0bb9f5886 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:17.285359 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c08348-8dcc-4b77-93ba-73e0bb9f5886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:17.285638 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c08348-8dcc-4b77-93ba-73e0bb9f5886 admin admin] PATH_INFO: `/v3/users/fed79b23091040daa06000754399eb84` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:17.286871 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c08348-8dcc-4b77-93ba-73e0bb9f5886 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fed79b23091040daa06000754399eb84)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:17.288833 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c08348-8dcc-4b77-93ba-73e0bb9f5886 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:17.486895 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e1c08348-8dcc-4b77-93ba-73e0bb9f5886 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fed79b23091040daa06000754399eb84'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:17.493885 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e1c08348-8dcc-4b77-93ba-73e0bb9f5886 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fed79b23091040daa06000754399eb84'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:17.494595 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e1c08348-8dcc-4b77-93ba-73e0bb9f5886 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fed79b23091040daa06000754399eb84'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:17.495215 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e1c08348-8dcc-4b77-93ba-73e0bb9f5886 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fed79b23091040daa06000754399eb84'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:17.508002 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e1c08348-8dcc-4b77-93ba-73e0bb9f5886 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fed79b23091040daa06000754399eb84'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:17.590636 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1418/2808] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 07:21:17 2022] DELETE /identity/v3/users/fed79b23091040daa06000754399eb84 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:21:17.837567 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d2779-4e8b-451c-95c7-4d20df7229df service placement] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:17.849163 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d2779-4e8b-451c-95c7-4d20df7229df service placement] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:17.912689 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86e271-6a74-47be-b7f4-fcec15fd3d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:17.913544 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86e271-6a74-47be-b7f4-fcec15fd3d90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:17.913865 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86e271-6a74-47be-b7f4-fcec15fd3d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:17.914145 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86e271-6a74-47be-b7f4-fcec15fd3d90 admin admin] PATH_INFO: `/v3/projects/c6c31a4ef7c3413aa6f7d2b9f08e4f63` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:17.915322 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b86e271-6a74-47be-b7f4-fcec15fd3d90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6c31a4ef7c3413aa6f7d2b9f08e4f63)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:17.917313 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b86e271-6a74-47be-b7f4-fcec15fd3d90 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:18.007825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1b86e271-6a74-47be-b7f4-fcec15fd3d90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6c31a4ef7c3413aa6f7d2b9f08e4f63', 'request_id': 'req-1b86e271-6a74-47be-b7f4-fcec15fd3d90'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:18.013752 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1b86e271-6a74-47be-b7f4-fcec15fd3d90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6c31a4ef7c3413aa6f7d2b9f08e4f63', 'request_id': 'req-1b86e271-6a74-47be-b7f4-fcec15fd3d90'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:18.075663 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1b86e271-6a74-47be-b7f4-fcec15fd3d90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6c31a4ef7c3413aa6f7d2b9f08e4f63', 'request_id': 'req-1b86e271-6a74-47be-b7f4-fcec15fd3d90'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:18.099298 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1b86e271-6a74-47be-b7f4-fcec15fd3d90 admin admin] The token cache is being invalidate because project c6c31a4ef7c3413aa6f7d2b9f08e4f63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 07:21:18.099529 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1b86e271-6a74-47be-b7f4-fcec15fd3d90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:18.100148 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1b86e271-6a74-47be-b7f4-fcec15fd3d90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:18.101694 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1391/2809] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 07:21:17 2022] DELETE /identity/v3/projects/c6c31a4ef7c3413aa6f7d2b9f08e4f63 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:21:18.106712 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01c9f9-b9e2-4b47-874a-753bf246d479 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:18.107669 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01c9f9-b9e2-4b47-874a-753bf246d479 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:18.107957 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01c9f9-b9e2-4b47-874a-753bf246d479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:18.153585 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-1d01c9f9-b9e2-4b47-874a-753bf246d479 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:21:18.155506 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-1d01c9f9-b9e2-4b47-874a-753bf246d479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:18.213253 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1419/2810] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:21:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:21:18.221641 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df86963-2862-41cb-ad10-d60bc94a8671 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:18.222150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df86963-2862-41cb-ad10-d60bc94a8671 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:18.222249 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df86963-2862-41cb-ad10-d60bc94a8671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:18.240203 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-5df86963-2862-41cb-ad10-d60bc94a8671 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:21:18.241455 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-5df86963-2862-41cb-ad10-d60bc94a8671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:18.249302 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1392/2811] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:21:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:21:18.253305 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01c9f9-b9e2-4b47-874a-753bf246d479 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:18.262796 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01c9f9-b9e2-4b47-874a-753bf246d479 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:18.267830 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a154c793-9ee2-49bb-adcb-c20f9181b1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:18.268507 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154c793-9ee2-49bb-adcb-c20f9181b1ff admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:18.268797 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154c793-9ee2-49bb-adcb-c20f9181b1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:18.269077 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154c793-9ee2-49bb-adcb-c20f9181b1ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:18.269799 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a154c793-9ee2-49bb-adcb-c20f9181b1ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 07:21:18.270319 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a154c793-9ee2-49bb-adcb-c20f9181b1ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:18.271689 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a154c793-9ee2-49bb-adcb-c20f9181b1ff admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:18.283606 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1420/2812] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 07:21:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 07:21:18.287779 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df86963-2862-41cb-ad10-d60bc94a8671 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:18.297076 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df86963-2862-41cb-ad10-d60bc94a8671 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:18.301876 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d150327-a9ce-4966-af25-cc06744f5292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:18.302528 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d150327-a9ce-4966-af25-cc06744f5292 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:18.302848 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d150327-a9ce-4966-af25-cc06744f5292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:18.303156 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d150327-a9ce-4966-af25-cc06744f5292 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:18.303845 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d150327-a9ce-4966-af25-cc06744f5292 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:18.305777 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d150327-a9ce-4966-af25-cc06744f5292 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:18.406808 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1393/2813] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 07:21:18 2022] POST /identity/v3/projects => generated 363 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 07:21:18.410770 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a154c793-9ee2-49bb-adcb-c20f9181b1ff admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:18.420910 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a154c793-9ee2-49bb-adcb-c20f9181b1ff admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:18.425535 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:21:18.478048 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc67ba-0abc-41a1-ac4e-1a1e82bd96a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:18.478713 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc67ba-0abc-41a1-ac4e-1a1e82bd96a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:18.479005 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc67ba-0abc-41a1-ac4e-1a1e82bd96a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:18.479288 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc67ba-0abc-41a1-ac4e-1a1e82bd96a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:18.480022 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc67ba-0abc-41a1-ac4e-1a1e82bd96a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:18.482034 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc67ba-0abc-41a1-ac4e-1a1e82bd96a4 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:18.555594 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1421/2814] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 07:21:18 2022] POST /identity/v3/users => generated 282 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 07:21:18.559759 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d150327-a9ce-4966-af25-cc06744f5292 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:18.570227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d150327-a9ce-4966-af25-cc06744f5292 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:18.575110 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeee024-d22e-4ba6-b493-9b41a8e8ce5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:18.576286 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeee024-d22e-4ba6-b493-9b41a8e8ce5d admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:18.576605 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeee024-d22e-4ba6-b493-9b41a8e8ce5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:18.576881 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeee024-d22e-4ba6-b493-9b41a8e8ce5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:18.577608 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeee024-d22e-4ba6-b493-9b41a8e8ce5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:18.579248 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeee024-d22e-4ba6-b493-9b41a8e8ce5d admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:18.589220 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1394/2815] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:21:18.594042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc67ba-0abc-41a1-ac4e-1a1e82bd96a4 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:18.604011 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc67ba-0abc-41a1-ac4e-1a1e82bd96a4 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:18.608868 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f541b5-ae2a-4447-a18c-07bee35fba25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:18.609518 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f541b5-ae2a-4447-a18c-07bee35fba25 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:18.609807 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f541b5-ae2a-4447-a18c-07bee35fba25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:18.610087 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f541b5-ae2a-4447-a18c-07bee35fba25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:18.610807 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f541b5-ae2a-4447-a18c-07bee35fba25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:18.612388 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f541b5-ae2a-4447-a18c-07bee35fba25 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:18.622148 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1422/2816] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:21:18.626573 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeee024-d22e-4ba6-b493-9b41a8e8ce5d admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:18.636677 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeee024-d22e-4ba6-b493-9b41a8e8ce5d admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:18.641882 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ffba9-739f-4af3-9711-b374d9cc57e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:18.642750 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ffba9-739f-4af3-9711-b374d9cc57e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:18.643038 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ffba9-739f-4af3-9711-b374d9cc57e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:18.643317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ffba9-739f-4af3-9711-b374d9cc57e9 admin admin] PATH_INFO: `/v3/projects/26aeb228307b440daccf6c00eb7ada14/users/259c4b210cd04c8d9f14a52cef7e8249/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:18.645583 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:21:18.659130 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12ffba9-739f-4af3-9711-b374d9cc57e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26aeb228307b440daccf6c00eb7ada14, user_id=259c4b210cd04c8d9f14a52cef7e8249, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:18.661771 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12ffba9-739f-4af3-9711-b374d9cc57e9 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:18.766477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1395/2817] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 07:21:18 2022] PUT /identity/v3/projects/26aeb228307b440daccf6c00eb7ada14/users/259c4b210cd04c8d9f14a52cef7e8249/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:21:18.776525 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ce618-d24c-4989-b6e9-c46dd4cfc9b0 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:18.776897 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ce618-d24c-4989-b6e9-c46dd4cfc9b0 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:18.777221 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ce618-d24c-4989-b6e9-c46dd4cfc9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:18.807879 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-742ce618-d24c-4989-b6e9-c46dd4cfc9b0 None None] MFA Rules not processed for user `259c4b210cd04c8d9f14a52cef7e8249`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:21:18.811956 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-742ce618-d24c-4989-b6e9-c46dd4cfc9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:18.877343 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1423/2818] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:21:18 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:21:18.882321 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ffba9-739f-4af3-9711-b374d9cc57e9 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:18.949727 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ffba9-739f-4af3-9711-b374d9cc57e9 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:18.955226 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f616cd3-ffa9-4d0f-8e32-569d8696e99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:18.955892 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f616cd3-ffa9-4d0f-8e32-569d8696e99e admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:18.956185 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f616cd3-ffa9-4d0f-8e32-569d8696e99e admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:18.956499 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f616cd3-ffa9-4d0f-8e32-569d8696e99e admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:18.957201 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f616cd3-ffa9-4d0f-8e32-569d8696e99e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:18.959104 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f616cd3-ffa9-4d0f-8e32-569d8696e99e admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:19.053268 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1396/2819] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 07:21:18 2022] POST /identity/v3/projects => generated 363 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 07:21:19.056891 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ce618-d24c-4989-b6e9-c46dd4cfc9b0 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:19.068203 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ce618-d24c-4989-b6e9-c46dd4cfc9b0 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:19.124397 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f022a84-844f-4228-a8ae-990adbaa9cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:19.125097 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f022a84-844f-4228-a8ae-990adbaa9cda admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:19.125386 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f022a84-844f-4228-a8ae-990adbaa9cda admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:19.125668 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f022a84-844f-4228-a8ae-990adbaa9cda admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:19.126359 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f022a84-844f-4228-a8ae-990adbaa9cda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:19.128295 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f022a84-844f-4228-a8ae-990adbaa9cda admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:19.221392 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1424/2820] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 07:21:19 2022] POST /identity/v3/users => generated 288 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 07:21:19.225499 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f616cd3-ffa9-4d0f-8e32-569d8696e99e admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:19.236888 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f616cd3-ffa9-4d0f-8e32-569d8696e99e admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:19.241757 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e73e3e-e28d-480d-b0c4-e64ac71bebb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:19.242416 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e73e3e-e28d-480d-b0c4-e64ac71bebb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:19.242726 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e73e3e-e28d-480d-b0c4-e64ac71bebb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:19.243034 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e73e3e-e28d-480d-b0c4-e64ac71bebb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:19.243768 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e73e3e-e28d-480d-b0c4-e64ac71bebb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:19.245399 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e73e3e-e28d-480d-b0c4-e64ac71bebb6 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:19.256997 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1397/2821] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 07:21:19.261179 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f022a84-844f-4228-a8ae-990adbaa9cda admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:19.271238 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f022a84-844f-4228-a8ae-990adbaa9cda admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:19.275903 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec33ee7-fa4c-4ac1-9597-9022c05008ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:19.276823 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec33ee7-fa4c-4ac1-9597-9022c05008ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:19.277111 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec33ee7-fa4c-4ac1-9597-9022c05008ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:19.277393 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec33ee7-fa4c-4ac1-9597-9022c05008ba admin admin] PATH_INFO: `/v3/projects/4d46561afae04fee8194ad67605e59d7/users/a218c5d79db04ed595eb2f7899406af4/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:19.279749 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:21:19.292524 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec33ee7-fa4c-4ac1-9597-9022c05008ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d46561afae04fee8194ad67605e59d7, user_id=a218c5d79db04ed595eb2f7899406af4, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:19.296011 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec33ee7-fa4c-4ac1-9597-9022c05008ba admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:19.358834 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1425/2822] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 07:21:19 2022] PUT /identity/v3/projects/4d46561afae04fee8194ad67605e59d7/users/a218c5d79db04ed595eb2f7899406af4/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:21:19.362940 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e73e3e-e28d-480d-b0c4-e64ac71bebb6 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:19.373402 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e73e3e-e28d-480d-b0c4-e64ac71bebb6 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:19.429393 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a01a11-0b5d-4996-8a5a-40d7c0cf603e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:19.430060 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a01a11-0b5d-4996-8a5a-40d7c0cf603e admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:19.430349 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a01a11-0b5d-4996-8a5a-40d7c0cf603e admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:19.430630 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a01a11-0b5d-4996-8a5a-40d7c0cf603e admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:19.431396 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a01a11-0b5d-4996-8a5a-40d7c0cf603e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:19.433032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a01a11-0b5d-4996-8a5a-40d7c0cf603e admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:19.442860 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1398/2823] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:21:19.447324 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec33ee7-fa4c-4ac1-9597-9022c05008ba admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:19.457556 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec33ee7-fa4c-4ac1-9597-9022c05008ba admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:19.462159 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f535f4a-60bb-4a5a-9bf9-5af4cef685d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:19.463033 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f535f4a-60bb-4a5a-9bf9-5af4cef685d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:19.463319 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f535f4a-60bb-4a5a-9bf9-5af4cef685d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:19.463595 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f535f4a-60bb-4a5a-9bf9-5af4cef685d5 admin admin] PATH_INFO: `/v3/projects/4d46561afae04fee8194ad67605e59d7/users/a218c5d79db04ed595eb2f7899406af4/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:19.465584 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f535f4a-60bb-4a5a-9bf9-5af4cef685d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d46561afae04fee8194ad67605e59d7, user_id=a218c5d79db04ed595eb2f7899406af4, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:19.468832 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f535f4a-60bb-4a5a-9bf9-5af4cef685d5 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:19.478838 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1426/2824] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 07:21:19 2022] PUT /identity/v3/projects/4d46561afae04fee8194ad67605e59d7/users/a218c5d79db04ed595eb2f7899406af4/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:21:19.490395 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce31e8-5f1b-42e1-853b-2d5d0e3e5fc2 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:19.490695 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce31e8-5f1b-42e1-853b-2d5d0e3e5fc2 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:19.490984 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce31e8-5f1b-42e1-853b-2d5d0e3e5fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:19.516596 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-28ce31e8-5f1b-42e1-853b-2d5d0e3e5fc2 None None] MFA Rules not processed for user `a218c5d79db04ed595eb2f7899406af4`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:21:19.517831 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-28ce31e8-5f1b-42e1-853b-2d5d0e3e5fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:19.559217 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:21:19.585309 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1399/2825] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:21:19 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:21:19.597626 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f535f4a-60bb-4a5a-9bf9-5af4cef685d5 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:19.599396 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-0f535f4a-60bb-4a5a-9bf9-5af4cef685d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:19.662802 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f535f4a-60bb-4a5a-9bf9-5af4cef685d5 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:19.666470 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9a4784-2167-42ff-9fc6-a5da93ce15fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:19.667121 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a4784-2167-42ff-9fc6-a5da93ce15fc service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:19.667411 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a4784-2167-42ff-9fc6-a5da93ce15fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:19.667694 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a4784-2167-42ff-9fc6-a5da93ce15fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:19.677615 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9a4784-2167-42ff-9fc6-a5da93ce15fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:19.679048 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9a4784-2167-42ff-9fc6-a5da93ce15fc service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:19.690413 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1427/2826] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:21:19 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:21:19.896748 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce31e8-5f1b-42e1-853b-2d5d0e3e5fc2 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:19.906268 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce31e8-5f1b-42e1-853b-2d5d0e3e5fc2 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:19.910991 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6790459-927b-4160-9789-6ffce2106a1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:19.910991 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6790459-927b-4160-9789-6ffce2106a1a service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:19.910991 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6790459-927b-4160-9789-6ffce2106a1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:19.910991 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6790459-927b-4160-9789-6ffce2106a1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:19.968780 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6790459-927b-4160-9789-6ffce2106a1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:19.970213 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6790459-927b-4160-9789-6ffce2106a1a service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:19.979158 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1400/2827] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:21:19 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 07:21:29.685458 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=38,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:21:29.975167 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=38 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:21:32.582329 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9a4784-2167-42ff-9fc6-a5da93ce15fc service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:32.644224 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9a4784-2167-42ff-9fc6-a5da93ce15fc service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:32.648747 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2fd34e-4fcd-4777-992c-d93b761388dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:32.649625 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2fd34e-4fcd-4777-992c-d93b761388dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:32.649911 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2fd34e-4fcd-4777-992c-d93b761388dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:32.650190 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2fd34e-4fcd-4777-992c-d93b761388dc admin admin] PATH_INFO: `/v3/users/259c4b210cd04c8d9f14a52cef7e8249` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:32.651382 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2fd34e-4fcd-4777-992c-d93b761388dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=259c4b210cd04c8d9f14a52cef7e8249)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:32.653407 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2fd34e-4fcd-4777-992c-d93b761388dc admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:32.869503 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ac2fd34e-4fcd-4777-992c-d93b761388dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '259c4b210cd04c8d9f14a52cef7e8249'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:32.877139 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ac2fd34e-4fcd-4777-992c-d93b761388dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '259c4b210cd04c8d9f14a52cef7e8249'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:32.877818 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ac2fd34e-4fcd-4777-992c-d93b761388dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '259c4b210cd04c8d9f14a52cef7e8249'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:32.878432 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ac2fd34e-4fcd-4777-992c-d93b761388dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '259c4b210cd04c8d9f14a52cef7e8249'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:32.891030 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ac2fd34e-4fcd-4777-992c-d93b761388dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '259c4b210cd04c8d9f14a52cef7e8249'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:33.002219 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1428/2828] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 07:21:32 2022] DELETE /identity/v3/users/259c4b210cd04c8d9f14a52cef7e8249 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:21:33.006818 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c50b3-76c4-4e03-acfb-0b03f5e19ff9 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:33.007275 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c50b3-76c4-4e03-acfb-0b03f5e19ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:33.007364 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c50b3-76c4-4e03-acfb-0b03f5e19ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:33.024988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-464c50b3-76c4-4e03-acfb-0b03f5e19ff9 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:21:33.026193 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-464c50b3-76c4-4e03-acfb-0b03f5e19ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:33.084674 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1401/2829] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:21:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:21:33.094436 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2fd34e-4fcd-4777-992c-d93b761388dc admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:33.095834 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ac2fd34e-4fcd-4777-992c-d93b761388dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:33.153088 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2fd34e-4fcd-4777-992c-d93b761388dc admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:33.156812 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6393b6-2553-461d-b262-e1421d4c24cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:33.157461 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6393b6-2553-461d-b262-e1421d4c24cb service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:33.157752 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6393b6-2553-461d-b262-e1421d4c24cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:33.158037 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6393b6-2553-461d-b262-e1421d4c24cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:33.167101 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6393b6-2553-461d-b262-e1421d4c24cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:33.168577 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6393b6-2553-461d-b262-e1421d4c24cb service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:33.178102 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1429/2830] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:21:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:21:33.574904 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c50b3-76c4-4e03-acfb-0b03f5e19ff9 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:33.576938 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-464c50b3-76c4-4e03-acfb-0b03f5e19ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:33.592265 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c50b3-76c4-4e03-acfb-0b03f5e19ff9 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:33.597075 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b1672f-1fcb-4283-8eb0-88c3081862cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:33.597998 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1672f-1fcb-4283-8eb0-88c3081862cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:33.598252 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1672f-1fcb-4283-8eb0-88c3081862cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:33.598537 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1672f-1fcb-4283-8eb0-88c3081862cb admin admin] PATH_INFO: `/v3/projects/26aeb228307b440daccf6c00eb7ada14` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:33.599772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b1672f-1fcb-4283-8eb0-88c3081862cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=26aeb228307b440daccf6c00eb7ada14)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:33.601762 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b1672f-1fcb-4283-8eb0-88c3081862cb admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:33.716507 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-25b1672f-1fcb-4283-8eb0-88c3081862cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26aeb228307b440daccf6c00eb7ada14', 'request_id': 'req-25b1672f-1fcb-4283-8eb0-88c3081862cb'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:33.722087 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-25b1672f-1fcb-4283-8eb0-88c3081862cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26aeb228307b440daccf6c00eb7ada14', 'request_id': 'req-25b1672f-1fcb-4283-8eb0-88c3081862cb'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:33.778339 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-25b1672f-1fcb-4283-8eb0-88c3081862cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26aeb228307b440daccf6c00eb7ada14', 'request_id': 'req-25b1672f-1fcb-4283-8eb0-88c3081862cb'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:33.789867 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-25b1672f-1fcb-4283-8eb0-88c3081862cb admin admin] The token cache is being invalidate because project 26aeb228307b440daccf6c00eb7ada14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 07:21:33.790162 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-25b1672f-1fcb-4283-8eb0-88c3081862cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:33.790794 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-25b1672f-1fcb-4283-8eb0-88c3081862cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:33.792403 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1402/2831] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 07:21:33 2022] DELETE /identity/v3/projects/26aeb228307b440daccf6c00eb7ada14 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:21:33.796525 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6393b6-2553-461d-b262-e1421d4c24cb service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:33.806135 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6393b6-2553-461d-b262-e1421d4c24cb service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:33.825276 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:21:33.864076 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df711df-0dce-406a-baa6-d370b346f3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:33.864996 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df711df-0dce-406a-baa6-d370b346f3d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:33.865290 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df711df-0dce-406a-baa6-d370b346f3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:33.865578 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df711df-0dce-406a-baa6-d370b346f3d7 admin admin] PATH_INFO: `/v3/users/a218c5d79db04ed595eb2f7899406af4` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:33.866772 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df711df-0dce-406a-baa6-d370b346f3d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a218c5d79db04ed595eb2f7899406af4)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:33.868730 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df711df-0dce-406a-baa6-d370b346f3d7 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:34.070430 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-2df711df-0dce-406a-baa6-d370b346f3d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a218c5d79db04ed595eb2f7899406af4'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:34.077939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-2df711df-0dce-406a-baa6-d370b346f3d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a218c5d79db04ed595eb2f7899406af4'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:34.078649 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-2df711df-0dce-406a-baa6-d370b346f3d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a218c5d79db04ed595eb2f7899406af4'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:34.079267 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-2df711df-0dce-406a-baa6-d370b346f3d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a218c5d79db04ed595eb2f7899406af4'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:34.090949 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-2df711df-0dce-406a-baa6-d370b346f3d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a218c5d79db04ed595eb2f7899406af4'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:34.193345 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1430/2832] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 07:21:33 2022] DELETE /identity/v3/users/a218c5d79db04ed595eb2f7899406af4 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:21:34.566050 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b1672f-1fcb-4283-8eb0-88c3081862cb admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:34.567573 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-25b1672f-1fcb-4283-8eb0-88c3081862cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:34.583682 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b1672f-1fcb-4283-8eb0-88c3081862cb admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:34.641910 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f3ae9d-93e2-445f-bede-2a5e4da69149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:34.642775 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f3ae9d-93e2-445f-bede-2a5e4da69149 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:34.643068 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f3ae9d-93e2-445f-bede-2a5e4da69149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:34.643348 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f3ae9d-93e2-445f-bede-2a5e4da69149 admin admin] PATH_INFO: `/v3/projects/4d46561afae04fee8194ad67605e59d7` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:34.644560 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f3ae9d-93e2-445f-bede-2a5e4da69149 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d46561afae04fee8194ad67605e59d7)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:34.646468 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f3ae9d-93e2-445f-bede-2a5e4da69149 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:34.729223 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:21:34.746284 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e7f3ae9d-93e2-445f-bede-2a5e4da69149 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d46561afae04fee8194ad67605e59d7', 'request_id': 'req-e7f3ae9d-93e2-445f-bede-2a5e4da69149'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:34.751843 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e7f3ae9d-93e2-445f-bede-2a5e4da69149 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d46561afae04fee8194ad67605e59d7', 'request_id': 'req-e7f3ae9d-93e2-445f-bede-2a5e4da69149'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:34.829847 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e7f3ae9d-93e2-445f-bede-2a5e4da69149 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d46561afae04fee8194ad67605e59d7', 'request_id': 'req-e7f3ae9d-93e2-445f-bede-2a5e4da69149'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:34.841438 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e7f3ae9d-93e2-445f-bede-2a5e4da69149 admin admin] The token cache is being invalidate because project 4d46561afae04fee8194ad67605e59d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 07:21:34.841779 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e7f3ae9d-93e2-445f-bede-2a5e4da69149 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:34.843341 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e7f3ae9d-93e2-445f-bede-2a5e4da69149 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:34.845235 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1403/2833] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 07:21:34 2022] DELETE /identity/v3/projects/4d46561afae04fee8194ad67605e59d7 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:21:34.850088 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d49e38-d29e-4e4a-a311-0ecb3d7dea25 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:34.850554 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d49e38-d29e-4e4a-a311-0ecb3d7dea25 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:34.850645 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d49e38-d29e-4e4a-a311-0ecb3d7dea25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:34.868055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-14d49e38-d29e-4e4a-a311-0ecb3d7dea25 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:21:34.869301 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-14d49e38-d29e-4e4a-a311-0ecb3d7dea25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:34.927999 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1431/2834] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:21:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:21:34.935671 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25dbc45-defe-417b-b8a2-ca755b6ca6b6 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:34.936586 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25dbc45-defe-417b-b8a2-ca755b6ca6b6 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:34.936886 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25dbc45-defe-417b-b8a2-ca755b6ca6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:34.955154 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-b25dbc45-defe-417b-b8a2-ca755b6ca6b6 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:21:34.956357 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b25dbc45-defe-417b-b8a2-ca755b6ca6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:34.964326 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1404/2835] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:21:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:21:34.968363 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d49e38-d29e-4e4a-a311-0ecb3d7dea25 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:34.978034 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d49e38-d29e-4e4a-a311-0ecb3d7dea25 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:34.982554 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55077a3-189c-4972-a6e8-875587a6d306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:34.984015 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55077a3-189c-4972-a6e8-875587a6d306 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:34.984404 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55077a3-189c-4972-a6e8-875587a6d306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:34.984913 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55077a3-189c-4972-a6e8-875587a6d306 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:34.986275 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55077a3-189c-4972-a6e8-875587a6d306 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 07:21:34.986850 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55077a3-189c-4972-a6e8-875587a6d306 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:34.988999 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55077a3-189c-4972-a6e8-875587a6d306 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:35.002046 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1432/2836] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 07:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 07:21:35.006354 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25dbc45-defe-417b-b8a2-ca755b6ca6b6 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:35.015717 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25dbc45-defe-417b-b8a2-ca755b6ca6b6 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:35.020384 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c9f75-067b-4ce3-8061-ab37dc0028f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:35.021057 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c9f75-067b-4ce3-8061-ab37dc0028f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:35.021347 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c9f75-067b-4ce3-8061-ab37dc0028f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:35.021630 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c9f75-067b-4ce3-8061-ab37dc0028f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:35.022332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66c9f75-067b-4ce3-8061-ab37dc0028f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:35.024221 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66c9f75-067b-4ce3-8061-ab37dc0028f9 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:35.134728 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1405/2837] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 07:21:35 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 07:21:35.138770 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55077a3-189c-4972-a6e8-875587a6d306 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:35.148985 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55077a3-189c-4972-a6e8-875587a6d306 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:35.175620 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:21:35.207898 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10acc72c-b969-4360-9c8f-4484e3129970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:35.208652 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acc72c-b969-4360-9c8f-4484e3129970 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:35.208938 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acc72c-b969-4360-9c8f-4484e3129970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:35.209218 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acc72c-b969-4360-9c8f-4484e3129970 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:35.209917 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10acc72c-b969-4360-9c8f-4484e3129970 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:35.211829 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10acc72c-b969-4360-9c8f-4484e3129970 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:35.281836 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1433/2838] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 07:21:35 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 07:21:35.286207 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c9f75-067b-4ce3-8061-ab37dc0028f9 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:35.297661 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c9f75-067b-4ce3-8061-ab37dc0028f9 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:35.302319 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b145c71-18b7-4efb-a1da-03dda67d5a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:35.303105 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b145c71-18b7-4efb-a1da-03dda67d5a42 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:35.303396 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b145c71-18b7-4efb-a1da-03dda67d5a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:35.303680 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b145c71-18b7-4efb-a1da-03dda67d5a42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:35.304439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b145c71-18b7-4efb-a1da-03dda67d5a42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:35.306035 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b145c71-18b7-4efb-a1da-03dda67d5a42 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:35.317105 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1406/2839] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:21:35.321239 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10acc72c-b969-4360-9c8f-4484e3129970 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:35.331257 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10acc72c-b969-4360-9c8f-4484e3129970 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:35.335987 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54806e1e-0c4f-442f-a5ce-cd7e8557aa12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:35.336686 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54806e1e-0c4f-442f-a5ce-cd7e8557aa12 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:35.336973 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54806e1e-0c4f-442f-a5ce-cd7e8557aa12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:35.337250 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54806e1e-0c4f-442f-a5ce-cd7e8557aa12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:35.337977 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54806e1e-0c4f-442f-a5ce-cd7e8557aa12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:35.339585 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54806e1e-0c4f-442f-a5ce-cd7e8557aa12 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:35.350087 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1434/2840] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 07:21:35.354068 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b145c71-18b7-4efb-a1da-03dda67d5a42 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:35.364724 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b145c71-18b7-4efb-a1da-03dda67d5a42 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:35.369211 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8a499-db07-4312-a356-1895ac481202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:35.370078 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8a499-db07-4312-a356-1895ac481202 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:35.370363 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8a499-db07-4312-a356-1895ac481202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:35.370639 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8a499-db07-4312-a356-1895ac481202 admin admin] PATH_INFO: `/v3/projects/213e658272a240a3b255c3958d3d7e33/users/e8afc626e16347669bdcbe1590974047/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:35.385005 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a8a499-db07-4312-a356-1895ac481202 admin admin] RBAC: Authorizing `identity:create_grant(project_id=213e658272a240a3b255c3958d3d7e33, user_id=e8afc626e16347669bdcbe1590974047, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:35.387591 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a8a499-db07-4312-a356-1895ac481202 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:35.453216 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1407/2841] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 07:21:35 2022] PUT /identity/v3/projects/213e658272a240a3b255c3958d3d7e33/users/e8afc626e16347669bdcbe1590974047/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:21:35.463680 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886350cc-54d3-487b-90e8-0e3be4a88114 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:35.463977 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886350cc-54d3-487b-90e8-0e3be4a88114 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:35.464263 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886350cc-54d3-487b-90e8-0e3be4a88114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:35.490897 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-886350cc-54d3-487b-90e8-0e3be4a88114 None None] MFA Rules not processed for user `e8afc626e16347669bdcbe1590974047`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:21:35.492114 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-886350cc-54d3-487b-90e8-0e3be4a88114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:35.553903 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1435/2842] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:21:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:21:35.558330 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8a499-db07-4312-a356-1895ac481202 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:35.619680 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8a499-db07-4312-a356-1895ac481202 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:35.624397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa1cc0d-57f3-4b46-b7c2-5bba2ef8c782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:35.625079 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1cc0d-57f3-4b46-b7c2-5bba2ef8c782 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:35.625370 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1cc0d-57f3-4b46-b7c2-5bba2ef8c782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:35.625656 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1cc0d-57f3-4b46-b7c2-5bba2ef8c782 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:35.626389 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa1cc0d-57f3-4b46-b7c2-5bba2ef8c782 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:35.628282 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa1cc0d-57f3-4b46-b7c2-5bba2ef8c782 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:35.754682 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1408/2843] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 07:21:35 2022] POST /identity/v3/projects => generated 385 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 07:21:35.758497 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886350cc-54d3-487b-90e8-0e3be4a88114 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:35.770074 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886350cc-54d3-487b-90e8-0e3be4a88114 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:35.832566 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10655ff5-90c6-49b4-ac85-814257b047a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:35.833244 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10655ff5-90c6-49b4-ac85-814257b047a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:35.833528 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10655ff5-90c6-49b4-ac85-814257b047a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:35.833808 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10655ff5-90c6-49b4-ac85-814257b047a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:35.834491 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10655ff5-90c6-49b4-ac85-814257b047a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:35.836510 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10655ff5-90c6-49b4-ac85-814257b047a0 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:35.959628 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1436/2844] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 07:21:35 2022] POST /identity/v3/users => generated 299 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 07:21:35.963462 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa1cc0d-57f3-4b46-b7c2-5bba2ef8c782 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:35.973744 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa1cc0d-57f3-4b46-b7c2-5bba2ef8c782 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:35.978409 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f04db5-1edb-4099-9dfa-e8c02068521b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:35.979058 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f04db5-1edb-4099-9dfa-e8c02068521b admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:35.979343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f04db5-1edb-4099-9dfa-e8c02068521b admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:35.979666 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f04db5-1edb-4099-9dfa-e8c02068521b admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:35.980393 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f04db5-1edb-4099-9dfa-e8c02068521b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:35.981996 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f04db5-1edb-4099-9dfa-e8c02068521b admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:35.993015 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1409/2845] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:21:35.997287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10655ff5-90c6-49b4-ac85-814257b047a0 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:36.008248 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10655ff5-90c6-49b4-ac85-814257b047a0 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:36.012821 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df551906-ee8f-41df-9a0c-a5011ead9b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:36.013727 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df551906-ee8f-41df-9a0c-a5011ead9b96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:36.014731 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df551906-ee8f-41df-9a0c-a5011ead9b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:36.015029 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df551906-ee8f-41df-9a0c-a5011ead9b96 admin admin] PATH_INFO: `/v3/projects/dffb5de8fa4d46378174f6eb20f8432f/users/142cdc2b0a27445194e9eefc85168ede/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:36.029687 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df551906-ee8f-41df-9a0c-a5011ead9b96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dffb5de8fa4d46378174f6eb20f8432f, user_id=142cdc2b0a27445194e9eefc85168ede, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:36.032323 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df551906-ee8f-41df-9a0c-a5011ead9b96 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:36.141493 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1437/2846] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 07:21:35 2022] PUT /identity/v3/projects/dffb5de8fa4d46378174f6eb20f8432f/users/142cdc2b0a27445194e9eefc85168ede/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:21:36.145540 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f04db5-1edb-4099-9dfa-e8c02068521b admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:36.155820 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f04db5-1edb-4099-9dfa-e8c02068521b admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:36.176348 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:21:36.214461 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb8360-e940-4939-b6d3-f37e40daa8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:36.215194 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb8360-e940-4939-b6d3-f37e40daa8bc admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:36.215487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb8360-e940-4939-b6d3-f37e40daa8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:36.215773 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb8360-e940-4939-b6d3-f37e40daa8bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:36.216544 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb8360-e940-4939-b6d3-f37e40daa8bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:36.218232 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb8360-e940-4939-b6d3-f37e40daa8bc admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:36.228382 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1410/2847] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:21:36 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:21:36.232341 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df551906-ee8f-41df-9a0c-a5011ead9b96 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:36.238674 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:21:36.244251 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df551906-ee8f-41df-9a0c-a5011ead9b96 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:36.249619 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef08cec2-cc63-4857-91cd-3c55b7374cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:36.250500 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08cec2-cc63-4857-91cd-3c55b7374cae admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:36.250787 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08cec2-cc63-4857-91cd-3c55b7374cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:36.251074 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08cec2-cc63-4857-91cd-3c55b7374cae admin admin] PATH_INFO: `/v3/projects/dffb5de8fa4d46378174f6eb20f8432f/users/142cdc2b0a27445194e9eefc85168ede/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:36.253109 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef08cec2-cc63-4857-91cd-3c55b7374cae admin admin] RBAC: Authorizing `identity:create_grant(project_id=dffb5de8fa4d46378174f6eb20f8432f, user_id=142cdc2b0a27445194e9eefc85168ede, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:36.255755 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef08cec2-cc63-4857-91cd-3c55b7374cae admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:36.266352 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1438/2848] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 07:21:36 2022] PUT /identity/v3/projects/dffb5de8fa4d46378174f6eb20f8432f/users/142cdc2b0a27445194e9eefc85168ede/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:21:36.277578 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48eeb8-ef1c-49fd-bf0f-59fa9d158735 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:36.278622 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48eeb8-ef1c-49fd-bf0f-59fa9d158735 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:36.278622 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48eeb8-ef1c-49fd-bf0f-59fa9d158735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:36.304919 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-4a48eeb8-ef1c-49fd-bf0f-59fa9d158735 None None] MFA Rules not processed for user `142cdc2b0a27445194e9eefc85168ede`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:21:36.306117 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4a48eeb8-ef1c-49fd-bf0f-59fa9d158735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:36.374785 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1411/2849] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:21:36 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:21:36.385818 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451304cd-3e3c-4343-a049-7eef41919f96 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:36.386114 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451304cd-3e3c-4343-a049-7eef41919f96 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:36.386391 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451304cd-3e3c-4343-a049-7eef41919f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:36.404064 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-451304cd-3e3c-4343-a049-7eef41919f96 None None] MFA Rules not processed for user `5af3a4a1bc1f4c4e9653d0a790f12606`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:21:36.405294 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-451304cd-3e3c-4343-a049-7eef41919f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:36.454927 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1439/2850] 10.222.0.40 () {58 vars in 1061 bytes} [Mon Jul 18 07:21:36 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:21:36.460807 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48eeb8-ef1c-49fd-bf0f-59fa9d158735 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:36.470277 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48eeb8-ef1c-49fd-bf0f-59fa9d158735 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:36.473841 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff3339-83e8-4750-8a71-402c7f13a213 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:36.474475 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff3339-83e8-4750-8a71-402c7f13a213 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:36.474833 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff3339-83e8-4750-8a71-402c7f13a213 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:36.475067 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff3339-83e8-4750-8a71-402c7f13a213 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:36.484617 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff3339-83e8-4750-8a71-402c7f13a213 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:36.486032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff3339-83e8-4750-8a71-402c7f13a213 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:36.495411 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1412/2851] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:21:36 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:21:36.522416 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451304cd-3e3c-4343-a049-7eef41919f96 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:36.587926 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451304cd-3e3c-4343-a049-7eef41919f96 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:36.593275 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ceb5f9-08ad-477d-afaa-3dfae4f6797e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:36.594144 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ceb5f9-08ad-477d-afaa-3dfae4f6797e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:36.594431 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ceb5f9-08ad-477d-afaa-3dfae4f6797e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:36.594710 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ceb5f9-08ad-477d-afaa-3dfae4f6797e admin admin] PATH_INFO: `/v3/users/e8afc626e16347669bdcbe1590974047` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:36.595925 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ceb5f9-08ad-477d-afaa-3dfae4f6797e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8afc626e16347669bdcbe1590974047)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:36.597919 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ceb5f9-08ad-477d-afaa-3dfae4f6797e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:36.780805 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-b1ceb5f9-08ad-477d-afaa-3dfae4f6797e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8afc626e16347669bdcbe1590974047'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:36.787667 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-b1ceb5f9-08ad-477d-afaa-3dfae4f6797e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8afc626e16347669bdcbe1590974047'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:36.788352 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-b1ceb5f9-08ad-477d-afaa-3dfae4f6797e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8afc626e16347669bdcbe1590974047'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:36.789006 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-b1ceb5f9-08ad-477d-afaa-3dfae4f6797e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8afc626e16347669bdcbe1590974047'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:36.801936 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-b1ceb5f9-08ad-477d-afaa-3dfae4f6797e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8afc626e16347669bdcbe1590974047'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:36.866768 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1440/2852] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 07:21:36 2022] DELETE /identity/v3/users/e8afc626e16347669bdcbe1590974047 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:21:36.871345 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5b780-50fd-4cd5-926c-effcb02cf3d2 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:36.871810 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5b780-50fd-4cd5-926c-effcb02cf3d2 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:36.871908 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5b780-50fd-4cd5-926c-effcb02cf3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:36.889213 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-1ca5b780-50fd-4cd5-926c-effcb02cf3d2 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:21:36.890446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-1ca5b780-50fd-4cd5-926c-effcb02cf3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:36.948150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1413/2853] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:21:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:21:36.957791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9baaa29-2ce2-4171-85ca-2d5fec2ea557 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:36.958086 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9baaa29-2ce2-4171-85ca-2d5fec2ea557 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:36.958369 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9baaa29-2ce2-4171-85ca-2d5fec2ea557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:36.976369 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-d9baaa29-2ce2-4171-85ca-2d5fec2ea557 None None] MFA Rules not processed for user `9824a96e2bcd4a16bfc6fe6bf4ea1ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:21:36.977595 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d9baaa29-2ce2-4171-85ca-2d5fec2ea557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:36.989747 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:21:37.030349 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1441/2854] 10.222.0.40 () {58 vars in 1062 bytes} [Mon Jul 18 07:21:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:21:37.036318 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca5b780-50fd-4cd5-926c-effcb02cf3d2 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:37.045596 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca5b780-50fd-4cd5-926c-effcb02cf3d2 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:37.049111 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2f523-03da-4198-83fc-e05d0ff4134e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:37.049741 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2f523-03da-4198-83fc-e05d0ff4134e service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:37.050042 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2f523-03da-4198-83fc-e05d0ff4134e service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:37.050298 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2f523-03da-4198-83fc-e05d0ff4134e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:37.059107 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a2f523-03da-4198-83fc-e05d0ff4134e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:37.060598 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a2f523-03da-4198-83fc-e05d0ff4134e service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:37.072394 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1414/2855] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:21:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:21:37.485567 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9baaa29-2ce2-4171-85ca-2d5fec2ea557 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:37.487009 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d9baaa29-2ce2-4171-85ca-2d5fec2ea557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:37.503624 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9baaa29-2ce2-4171-85ca-2d5fec2ea557 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:37.508365 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b31483-1c22-4b84-89ef-0424b79879f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:37.509407 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b31483-1c22-4b84-89ef-0424b79879f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:37.509717 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b31483-1c22-4b84-89ef-0424b79879f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:37.510012 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b31483-1c22-4b84-89ef-0424b79879f7 admin admin] PATH_INFO: `/v3/projects/213e658272a240a3b255c3958d3d7e33` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:37.511173 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b31483-1c22-4b84-89ef-0424b79879f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=213e658272a240a3b255c3958d3d7e33)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:37.513132 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b31483-1c22-4b84-89ef-0424b79879f7 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:37.618550 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-52b31483-1c22-4b84-89ef-0424b79879f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '213e658272a240a3b255c3958d3d7e33', 'request_id': 'req-52b31483-1c22-4b84-89ef-0424b79879f7'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:37.625653 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-52b31483-1c22-4b84-89ef-0424b79879f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '213e658272a240a3b255c3958d3d7e33', 'request_id': 'req-52b31483-1c22-4b84-89ef-0424b79879f7'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:37.702029 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-52b31483-1c22-4b84-89ef-0424b79879f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '213e658272a240a3b255c3958d3d7e33', 'request_id': 'req-52b31483-1c22-4b84-89ef-0424b79879f7'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:37.713033 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-52b31483-1c22-4b84-89ef-0424b79879f7 admin admin] The token cache is being invalidate because project 213e658272a240a3b255c3958d3d7e33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 07:21:37.713353 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-52b31483-1c22-4b84-89ef-0424b79879f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:37.714043 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-52b31483-1c22-4b84-89ef-0424b79879f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:37.715528 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1442/2856] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 07:21:37 2022] DELETE /identity/v3/projects/213e658272a240a3b255c3958d3d7e33 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 07:21:37.719697 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2f523-03da-4198-83fc-e05d0ff4134e service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:37.729663 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2f523-03da-4198-83fc-e05d0ff4134e service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:37.746428 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:21:37.786382 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6600e19-f2ec-4ea0-b4c8-1e2050689d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:37.787264 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6600e19-f2ec-4ea0-b4c8-1e2050689d81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:37.787552 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6600e19-f2ec-4ea0-b4c8-1e2050689d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:37.787834 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6600e19-f2ec-4ea0-b4c8-1e2050689d81 admin admin] PATH_INFO: `/v3/users/142cdc2b0a27445194e9eefc85168ede` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:37.789079 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6600e19-f2ec-4ea0-b4c8-1e2050689d81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=142cdc2b0a27445194e9eefc85168ede)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:37.791079 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6600e19-f2ec-4ea0-b4c8-1e2050689d81 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:37.971302 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-c6600e19-f2ec-4ea0-b4c8-1e2050689d81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '142cdc2b0a27445194e9eefc85168ede'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:37.978486 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-c6600e19-f2ec-4ea0-b4c8-1e2050689d81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '142cdc2b0a27445194e9eefc85168ede'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:37.991360 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-c6600e19-f2ec-4ea0-b4c8-1e2050689d81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '142cdc2b0a27445194e9eefc85168ede'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:37.992046 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-c6600e19-f2ec-4ea0-b4c8-1e2050689d81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '142cdc2b0a27445194e9eefc85168ede'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:37.992745 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-c6600e19-f2ec-4ea0-b4c8-1e2050689d81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '142cdc2b0a27445194e9eefc85168ede'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:38.067337 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1415/2857] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 07:21:37 2022] DELETE /identity/v3/users/142cdc2b0a27445194e9eefc85168ede => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:21:38.457279 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b31483-1c22-4b84-89ef-0424b79879f7 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:38.458809 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-52b31483-1c22-4b84-89ef-0424b79879f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:38.474806 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b31483-1c22-4b84-89ef-0424b79879f7 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:38.534198 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4d4247-bae9-42a7-8ffd-efff11342d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:38.535067 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d4247-bae9-42a7-8ffd-efff11342d65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:38.535356 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d4247-bae9-42a7-8ffd-efff11342d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:38.535638 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d4247-bae9-42a7-8ffd-efff11342d65 admin admin] PATH_INFO: `/v3/projects/dffb5de8fa4d46378174f6eb20f8432f` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:38.536862 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4d4247-bae9-42a7-8ffd-efff11342d65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dffb5de8fa4d46378174f6eb20f8432f)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:38.538945 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4d4247-bae9-42a7-8ffd-efff11342d65 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:38.647335 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4a4d4247-bae9-42a7-8ffd-efff11342d65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dffb5de8fa4d46378174f6eb20f8432f', 'request_id': 'req-4a4d4247-bae9-42a7-8ffd-efff11342d65'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:38.649199 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:21:38.653721 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4a4d4247-bae9-42a7-8ffd-efff11342d65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dffb5de8fa4d46378174f6eb20f8432f', 'request_id': 'req-4a4d4247-bae9-42a7-8ffd-efff11342d65'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:38.744216 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4a4d4247-bae9-42a7-8ffd-efff11342d65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dffb5de8fa4d46378174f6eb20f8432f', 'request_id': 'req-4a4d4247-bae9-42a7-8ffd-efff11342d65'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:38.756505 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4a4d4247-bae9-42a7-8ffd-efff11342d65 admin admin] The token cache is being invalidate because project dffb5de8fa4d46378174f6eb20f8432f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 07:21:38.756800 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4a4d4247-bae9-42a7-8ffd-efff11342d65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:38.757452 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4a4d4247-bae9-42a7-8ffd-efff11342d65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:38.759006 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1443/2858] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 07:21:38 2022] DELETE /identity/v3/projects/dffb5de8fa4d46378174f6eb20f8432f => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:21:38.764457 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d33dd-3ba6-4a5e-b854-6855ea865a27 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:38.764758 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d33dd-3ba6-4a5e-b854-6855ea865a27 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:38.765037 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d33dd-3ba6-4a5e-b854-6855ea865a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:38.783401 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-641d33dd-3ba6-4a5e-b854-6855ea865a27 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:21:38.791756 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-641d33dd-3ba6-4a5e-b854-6855ea865a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:38.845553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1416/2859] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:21:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 07:21:38.853388 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2f2c2-2518-4fcf-b511-0e56d9996c52 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:38.853715 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2f2c2-2518-4fcf-b511-0e56d9996c52 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:38.854028 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2f2c2-2518-4fcf-b511-0e56d9996c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:38.872434 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-ffa2f2c2-2518-4fcf-b511-0e56d9996c52 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:21:38.874318 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ffa2f2c2-2518-4fcf-b511-0e56d9996c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:38.882430 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1444/2860] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:21:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 07:21:38.886913 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641d33dd-3ba6-4a5e-b854-6855ea865a27 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:38.897575 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641d33dd-3ba6-4a5e-b854-6855ea865a27 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:38.902119 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364ae02-7f93-4eb3-ad34-40e5f18b77e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:38.902770 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364ae02-7f93-4eb3-ad34-40e5f18b77e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:38.903065 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364ae02-7f93-4eb3-ad34-40e5f18b77e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:38.903343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364ae02-7f93-4eb3-ad34-40e5f18b77e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:38.904072 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3364ae02-7f93-4eb3-ad34-40e5f18b77e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 07:21:38.904443 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3364ae02-7f93-4eb3-ad34-40e5f18b77e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:38.906233 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3364ae02-7f93-4eb3-ad34-40e5f18b77e7 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:38.917714 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1417/2861] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 07:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 07:21:38.922004 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa2f2c2-2518-4fcf-b511-0e56d9996c52 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:38.931261 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa2f2c2-2518-4fcf-b511-0e56d9996c52 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:38.936767 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf5482-4336-4164-9d17-72bbb8a163e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:38.937508 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf5482-4336-4164-9d17-72bbb8a163e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:38.937803 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf5482-4336-4164-9d17-72bbb8a163e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:38.938088 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf5482-4336-4164-9d17-72bbb8a163e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:38.938778 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cf5482-4336-4164-9d17-72bbb8a163e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:38.940705 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cf5482-4336-4164-9d17-72bbb8a163e9 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:39.039751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1445/2862] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 07:21:38 2022] POST /identity/v3/projects => generated 387 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 07:21:39.043819 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364ae02-7f93-4eb3-ad34-40e5f18b77e7 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:39.054520 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364ae02-7f93-4eb3-ad34-40e5f18b77e7 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:39.078953 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:21:39.111826 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09878e52-cb32-456f-973b-10c7f706db28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:39.112517 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09878e52-cb32-456f-973b-10c7f706db28 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:39.112807 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09878e52-cb32-456f-973b-10c7f706db28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:39.113087 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09878e52-cb32-456f-973b-10c7f706db28 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:39.113767 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09878e52-cb32-456f-973b-10c7f706db28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:39.115762 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09878e52-cb32-456f-973b-10c7f706db28 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:39.215526 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1418/2863] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 07:21:39 2022] POST /identity/v3/users => generated 294 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 07:21:39.220716 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf5482-4336-4164-9d17-72bbb8a163e9 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:39.231293 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf5482-4336-4164-9d17-72bbb8a163e9 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:39.236093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b532a2-e230-4678-949b-0bf346ae2a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:39.236791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b532a2-e230-4678-949b-0bf346ae2a53 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:39.237083 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b532a2-e230-4678-949b-0bf346ae2a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:39.237362 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b532a2-e230-4678-949b-0bf346ae2a53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:39.238088 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b532a2-e230-4678-949b-0bf346ae2a53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:39.239688 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b532a2-e230-4678-949b-0bf346ae2a53 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:39.250344 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1446/2864] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:21:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:21:39.254280 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09878e52-cb32-456f-973b-10c7f706db28 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:39.264813 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09878e52-cb32-456f-973b-10c7f706db28 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:39.269463 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8bda5b-d059-4ddd-81c1-253c4009ab3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:39.270118 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bda5b-d059-4ddd-81c1-253c4009ab3d admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:39.270408 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bda5b-d059-4ddd-81c1-253c4009ab3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:39.270691 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bda5b-d059-4ddd-81c1-253c4009ab3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:39.271419 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8bda5b-d059-4ddd-81c1-253c4009ab3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:39.273050 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8bda5b-d059-4ddd-81c1-253c4009ab3d admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:39.283780 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1419/2865] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:21:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:21:39.287909 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b532a2-e230-4678-949b-0bf346ae2a53 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:39.298070 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b532a2-e230-4678-949b-0bf346ae2a53 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:39.302603 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e36da-c78f-4919-a91f-bf8135c8af07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:39.303469 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e36da-c78f-4919-a91f-bf8135c8af07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:39.303749 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e36da-c78f-4919-a91f-bf8135c8af07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:39.304024 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e36da-c78f-4919-a91f-bf8135c8af07 admin admin] PATH_INFO: `/v3/projects/270a1501e3b749e691b55d5779a5c788/users/79b02f1f9cbf4ff0a919e29f29c968a5/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:39.318167 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0e36da-c78f-4919-a91f-bf8135c8af07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=270a1501e3b749e691b55d5779a5c788, user_id=79b02f1f9cbf4ff0a919e29f29c968a5, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:39.321094 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0e36da-c78f-4919-a91f-bf8135c8af07 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:39.424365 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1447/2866] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 07:21:39 2022] PUT /identity/v3/projects/270a1501e3b749e691b55d5779a5c788/users/79b02f1f9cbf4ff0a919e29f29c968a5/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:21:39.434485 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb3a2f-fd01-49e1-8662-3d0df78516ba None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:39.434784 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb3a2f-fd01-49e1-8662-3d0df78516ba None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:39.435069 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb3a2f-fd01-49e1-8662-3d0df78516ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:39.461929 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-2edb3a2f-fd01-49e1-8662-3d0df78516ba None None] MFA Rules not processed for user `79b02f1f9cbf4ff0a919e29f29c968a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:21:39.463163 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-2edb3a2f-fd01-49e1-8662-3d0df78516ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:39.525644 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1420/2867] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:21:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:21:39.530604 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e36da-c78f-4919-a91f-bf8135c8af07 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:39.591016 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e36da-c78f-4919-a91f-bf8135c8af07 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:39.595771 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de6f179-a0b6-4340-a4aa-12ec28512b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:39.596460 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6f179-a0b6-4340-a4aa-12ec28512b28 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:39.596755 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6f179-a0b6-4340-a4aa-12ec28512b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:39.597037 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6f179-a0b6-4340-a4aa-12ec28512b28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:39.597726 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de6f179-a0b6-4340-a4aa-12ec28512b28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:39.599611 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de6f179-a0b6-4340-a4aa-12ec28512b28 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:39.682872 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1448/2868] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 07:21:39 2022] POST /identity/v3/projects => generated 389 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 07:21:39.687552 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb3a2f-fd01-49e1-8662-3d0df78516ba None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:39.698767 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb3a2f-fd01-49e1-8662-3d0df78516ba None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:39.753660 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14c90d-229e-4fd2-8770-8c0ed9ccb989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:39.754326 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14c90d-229e-4fd2-8770-8c0ed9ccb989 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:39.754612 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14c90d-229e-4fd2-8770-8c0ed9ccb989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:39.754891 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14c90d-229e-4fd2-8770-8c0ed9ccb989 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:39.755587 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a14c90d-229e-4fd2-8770-8c0ed9ccb989 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:39.757533 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a14c90d-229e-4fd2-8770-8c0ed9ccb989 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:39.847113 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1421/2869] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 07:21:39 2022] POST /identity/v3/users => generated 301 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 07:21:39.851052 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de6f179-a0b6-4340-a4aa-12ec28512b28 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:39.862042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de6f179-a0b6-4340-a4aa-12ec28512b28 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:39.866792 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cdc316-89d0-46b8-b6bd-bb5749c3c988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:39.867454 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cdc316-89d0-46b8-b6bd-bb5749c3c988 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:39.867783 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cdc316-89d0-46b8-b6bd-bb5749c3c988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:39.868073 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cdc316-89d0-46b8-b6bd-bb5749c3c988 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:39.869037 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cdc316-89d0-46b8-b6bd-bb5749c3c988 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:39.870641 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cdc316-89d0-46b8-b6bd-bb5749c3c988 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:39.882168 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1449/2870] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:21:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:21:39.886295 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14c90d-229e-4fd2-8770-8c0ed9ccb989 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:39.896865 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14c90d-229e-4fd2-8770-8c0ed9ccb989 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:39.901386 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8233af-b667-4ed0-9605-dc99c39fbf88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:39.902251 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8233af-b667-4ed0-9605-dc99c39fbf88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:39.902532 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8233af-b667-4ed0-9605-dc99c39fbf88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:39.902805 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8233af-b667-4ed0-9605-dc99c39fbf88 admin admin] PATH_INFO: `/v3/projects/eb28001717a4460c8441d03df12c46fa/users/fafbf75a3d5847c58a83235e270af3b2/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:39.917555 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8233af-b667-4ed0-9605-dc99c39fbf88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb28001717a4460c8441d03df12c46fa, user_id=fafbf75a3d5847c58a83235e270af3b2, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:39.920307 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8233af-b667-4ed0-9605-dc99c39fbf88 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:39.976751 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:21:39.981377 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1422/2871] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 07:21:39 2022] PUT /identity/v3/projects/eb28001717a4460c8441d03df12c46fa/users/fafbf75a3d5847c58a83235e270af3b2/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:21:39.986306 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cdc316-89d0-46b8-b6bd-bb5749c3c988 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:39.996635 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cdc316-89d0-46b8-b6bd-bb5749c3c988 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:40.043783 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:21:40.053989 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9c0b2-5246-4d53-bd43-adef5cbc55ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:40.054664 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9c0b2-5246-4d53-bd43-adef5cbc55ad admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:40.054955 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9c0b2-5246-4d53-bd43-adef5cbc55ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:40.055247 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9c0b2-5246-4d53-bd43-adef5cbc55ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:40.055969 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e9c0b2-5246-4d53-bd43-adef5cbc55ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:40.057616 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e9c0b2-5246-4d53-bd43-adef5cbc55ad admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:40.068177 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1450/2872] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:21:39 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:21:40.072613 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8233af-b667-4ed0-9605-dc99c39fbf88 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:40.083372 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8233af-b667-4ed0-9605-dc99c39fbf88 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:40.087917 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aeefaf-c9c0-4030-888c-547735208eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:40.088918 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aeefaf-c9c0-4030-888c-547735208eb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:40.089207 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aeefaf-c9c0-4030-888c-547735208eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:40.089490 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aeefaf-c9c0-4030-888c-547735208eb0 admin admin] PATH_INFO: `/v3/projects/eb28001717a4460c8441d03df12c46fa/users/fafbf75a3d5847c58a83235e270af3b2/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:40.091470 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aeefaf-c9c0-4030-888c-547735208eb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb28001717a4460c8441d03df12c46fa, user_id=fafbf75a3d5847c58a83235e270af3b2, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:40.094276 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aeefaf-c9c0-4030-888c-547735208eb0 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:40.106008 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1423/2873] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 07:21:40 2022] PUT /identity/v3/projects/eb28001717a4460c8441d03df12c46fa/users/fafbf75a3d5847c58a83235e270af3b2/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 07:21:40.116550 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a219849-a30e-465a-b15f-75cf5e6b6bdc None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:40.116948 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a219849-a30e-465a-b15f-75cf5e6b6bdc None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:40.117218 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a219849-a30e-465a-b15f-75cf5e6b6bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:40.145857 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-5a219849-a30e-465a-b15f-75cf5e6b6bdc None None] MFA Rules not processed for user `fafbf75a3d5847c58a83235e270af3b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:21:40.147125 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-5a219849-a30e-465a-b15f-75cf5e6b6bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:40.217549 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1451/2874] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:21:40 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:21:40.228341 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aeefaf-c9c0-4030-888c-547735208eb0 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:40.230059 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-31aeefaf-c9c0-4030-888c-547735208eb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:40.286451 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aeefaf-c9c0-4030-888c-547735208eb0 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:40.290118 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9884e3-107d-4fec-bd74-3cd3aa6494ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:40.290752 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9884e3-107d-4fec-bd74-3cd3aa6494ac service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:40.291040 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9884e3-107d-4fec-bd74-3cd3aa6494ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:40.291321 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9884e3-107d-4fec-bd74-3cd3aa6494ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:40.301175 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9884e3-107d-4fec-bd74-3cd3aa6494ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:40.302585 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9884e3-107d-4fec-bd74-3cd3aa6494ac service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:40.312528 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1424/2875] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:21:40 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:21:40.439058 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a219849-a30e-465a-b15f-75cf5e6b6bdc None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:40.501821 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a219849-a30e-465a-b15f-75cf5e6b6bdc None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:40.506369 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71117877-348a-4bc8-8231-e981541c47ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:40.507238 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71117877-348a-4bc8-8231-e981541c47ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:40.507527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71117877-348a-4bc8-8231-e981541c47ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:40.507805 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71117877-348a-4bc8-8231-e981541c47ac admin admin] PATH_INFO: `/v3/users/79b02f1f9cbf4ff0a919e29f29c968a5` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:40.509051 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71117877-348a-4bc8-8231-e981541c47ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=79b02f1f9cbf4ff0a919e29f29c968a5)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:40.510981 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71117877-348a-4bc8-8231-e981541c47ac admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:40.639484 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-71117877-348a-4bc8-8231-e981541c47ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79b02f1f9cbf4ff0a919e29f29c968a5'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:40.646566 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-71117877-348a-4bc8-8231-e981541c47ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79b02f1f9cbf4ff0a919e29f29c968a5'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:40.647232 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-71117877-348a-4bc8-8231-e981541c47ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79b02f1f9cbf4ff0a919e29f29c968a5'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:40.647856 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-71117877-348a-4bc8-8231-e981541c47ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79b02f1f9cbf4ff0a919e29f29c968a5'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:40.660668 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-71117877-348a-4bc8-8231-e981541c47ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79b02f1f9cbf4ff0a919e29f29c968a5'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:40.662312 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:21:40.724201 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1452/2876] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 07:21:40 2022] DELETE /identity/v3/users/79b02f1f9cbf4ff0a919e29f29c968a5 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:21:40.728764 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a11121a-413b-4b66-b856-c34afe7e1eaa None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:40.729065 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a11121a-413b-4b66-b856-c34afe7e1eaa None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:40.729354 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a11121a-413b-4b66-b856-c34afe7e1eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:40.748341 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-1a11121a-413b-4b66-b856-c34afe7e1eaa None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:21:40.749628 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-1a11121a-413b-4b66-b856-c34afe7e1eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:40.930468 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1425/2877] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:21:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:21:40.940160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71117877-348a-4bc8-8231-e981541c47ac admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:40.941608 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-71117877-348a-4bc8-8231-e981541c47ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:40.997665 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71117877-348a-4bc8-8231-e981541c47ac admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:41.001362 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c261de-9e2d-4213-b1a0-4ecc59e795cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:41.002004 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c261de-9e2d-4213-b1a0-4ecc59e795cf service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:41.002293 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c261de-9e2d-4213-b1a0-4ecc59e795cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:41.002574 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c261de-9e2d-4213-b1a0-4ecc59e795cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:41.011417 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c261de-9e2d-4213-b1a0-4ecc59e795cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:41.012886 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c261de-9e2d-4213-b1a0-4ecc59e795cf service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:41.022672 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1453/2878] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:21:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:21:41.431562 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a11121a-413b-4b66-b856-c34afe7e1eaa None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:41.433049 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-1a11121a-413b-4b66-b856-c34afe7e1eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:41.446071 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a11121a-413b-4b66-b856-c34afe7e1eaa None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:41.450678 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3503e60b-64bf-44a2-bd4e-7a39fbb78032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:41.452138 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503e60b-64bf-44a2-bd4e-7a39fbb78032 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:41.452432 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503e60b-64bf-44a2-bd4e-7a39fbb78032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:41.452804 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503e60b-64bf-44a2-bd4e-7a39fbb78032 admin admin] PATH_INFO: `/v3/projects/270a1501e3b749e691b55d5779a5c788` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:41.454126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3503e60b-64bf-44a2-bd4e-7a39fbb78032 admin admin] RBAC: Authorizing `identity:delete_project(project_id=270a1501e3b749e691b55d5779a5c788)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:41.456806 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3503e60b-64bf-44a2-bd4e-7a39fbb78032 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:41.570726 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3503e60b-64bf-44a2-bd4e-7a39fbb78032 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '270a1501e3b749e691b55d5779a5c788', 'request_id': 'req-3503e60b-64bf-44a2-bd4e-7a39fbb78032'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:41.576303 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3503e60b-64bf-44a2-bd4e-7a39fbb78032 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '270a1501e3b749e691b55d5779a5c788', 'request_id': 'req-3503e60b-64bf-44a2-bd4e-7a39fbb78032'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:41.643857 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3503e60b-64bf-44a2-bd4e-7a39fbb78032 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '270a1501e3b749e691b55d5779a5c788', 'request_id': 'req-3503e60b-64bf-44a2-bd4e-7a39fbb78032'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:41.655943 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3503e60b-64bf-44a2-bd4e-7a39fbb78032 admin admin] The token cache is being invalidate because project 270a1501e3b749e691b55d5779a5c788 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 07:21:41.656232 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3503e60b-64bf-44a2-bd4e-7a39fbb78032 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:41.656904 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3503e60b-64bf-44a2-bd4e-7a39fbb78032 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:41.658448 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1426/2879] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 07:21:41 2022] DELETE /identity/v3/projects/270a1501e3b749e691b55d5779a5c788 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:21:41.662405 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c261de-9e2d-4213-b1a0-4ecc59e795cf service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:41.672879 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c261de-9e2d-4213-b1a0-4ecc59e795cf service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:41.730582 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d8911a-73ed-4b50-b5c2-e8856869c793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:41.731366 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d8911a-73ed-4b50-b5c2-e8856869c793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:41.731650 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d8911a-73ed-4b50-b5c2-e8856869c793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:41.731926 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d8911a-73ed-4b50-b5c2-e8856869c793 admin admin] PATH_INFO: `/v3/users/fafbf75a3d5847c58a83235e270af3b2` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:41.733664 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d8911a-73ed-4b50-b5c2-e8856869c793 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fafbf75a3d5847c58a83235e270af3b2)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:41.735565 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d8911a-73ed-4b50-b5c2-e8856869c793 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:41.970926 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-17d8911a-73ed-4b50-b5c2-e8856869c793 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fafbf75a3d5847c58a83235e270af3b2'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:41.979571 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-17d8911a-73ed-4b50-b5c2-e8856869c793 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fafbf75a3d5847c58a83235e270af3b2'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:41.980276 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-17d8911a-73ed-4b50-b5c2-e8856869c793 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fafbf75a3d5847c58a83235e270af3b2'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:41.981069 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-17d8911a-73ed-4b50-b5c2-e8856869c793 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fafbf75a3d5847c58a83235e270af3b2'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:41.993766 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-17d8911a-73ed-4b50-b5c2-e8856869c793 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fafbf75a3d5847c58a83235e270af3b2'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:42.063741 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1454/2880] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 07:21:41 2022] DELETE /identity/v3/users/fafbf75a3d5847c58a83235e270af3b2 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:21:42.723966 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3503e60b-64bf-44a2-bd4e-7a39fbb78032 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:42.725673 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-3503e60b-64bf-44a2-bd4e-7a39fbb78032 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:42.741291 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3503e60b-64bf-44a2-bd4e-7a39fbb78032 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:42.751831 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:21:42.799433 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e33316-2980-4eb5-9878-b0b1e0e71bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:42.800347 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e33316-2980-4eb5-9878-b0b1e0e71bce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:42.800659 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e33316-2980-4eb5-9878-b0b1e0e71bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:42.800956 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e33316-2980-4eb5-9878-b0b1e0e71bce admin admin] PATH_INFO: `/v3/projects/eb28001717a4460c8441d03df12c46fa` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:42.802175 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e33316-2980-4eb5-9878-b0b1e0e71bce admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb28001717a4460c8441d03df12c46fa)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:42.804055 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e33316-2980-4eb5-9878-b0b1e0e71bce admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:42.904112 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-95e33316-2980-4eb5-9878-b0b1e0e71bce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb28001717a4460c8441d03df12c46fa', 'request_id': 'req-95e33316-2980-4eb5-9878-b0b1e0e71bce'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:42.910269 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-95e33316-2980-4eb5-9878-b0b1e0e71bce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb28001717a4460c8441d03df12c46fa', 'request_id': 'req-95e33316-2980-4eb5-9878-b0b1e0e71bce'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:42.995437 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-95e33316-2980-4eb5-9878-b0b1e0e71bce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb28001717a4460c8441d03df12c46fa', 'request_id': 'req-95e33316-2980-4eb5-9878-b0b1e0e71bce'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:43.009575 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-95e33316-2980-4eb5-9878-b0b1e0e71bce admin admin] The token cache is being invalidate because project eb28001717a4460c8441d03df12c46fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 07:21:43.009866 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-95e33316-2980-4eb5-9878-b0b1e0e71bce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:43.010487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-95e33316-2980-4eb5-9878-b0b1e0e71bce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:43.012032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1427/2881] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 07:21:42 2022] DELETE /identity/v3/projects/eb28001717a4460c8441d03df12c46fa => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:21:43.017030 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e92e4-b57d-4cb4-a3f5-533a43ecd546 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:43.017512 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e92e4-b57d-4cb4-a3f5-533a43ecd546 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:43.017651 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e92e4-b57d-4cb4-a3f5-533a43ecd546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:43.035619 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-445e92e4-b57d-4cb4-a3f5-533a43ecd546 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:21:43.036862 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-445e92e4-b57d-4cb4-a3f5-533a43ecd546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:43.046191 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:21:43.097539 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1455/2882] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:21:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:21:43.105658 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af82b2-56ca-4e59-8793-c948f565402e None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:43.105947 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af82b2-56ca-4e59-8793-c948f565402e None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:43.106229 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af82b2-56ca-4e59-8793-c948f565402e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:43.125254 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-e1af82b2-56ca-4e59-8793-c948f565402e None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:21:43.126452 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e1af82b2-56ca-4e59-8793-c948f565402e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:43.134238 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1428/2883] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:21:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:21:43.138355 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e92e4-b57d-4cb4-a3f5-533a43ecd546 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:43.148279 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e92e4-b57d-4cb4-a3f5-533a43ecd546 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:43.152945 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d22604-e62b-46df-b4e7-9b05234bdaba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:43.153601 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d22604-e62b-46df-b4e7-9b05234bdaba admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:43.153890 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d22604-e62b-46df-b4e7-9b05234bdaba admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:43.154172 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d22604-e62b-46df-b4e7-9b05234bdaba admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:43.155181 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d22604-e62b-46df-b4e7-9b05234bdaba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 07:21:43.155497 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d22604-e62b-46df-b4e7-9b05234bdaba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:43.157158 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d22604-e62b-46df-b4e7-9b05234bdaba admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:43.168588 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1456/2884] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 07:21:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 07:21:43.173427 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af82b2-56ca-4e59-8793-c948f565402e None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:43.182859 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af82b2-56ca-4e59-8793-c948f565402e None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:43.188201 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a2a8a-7f4f-4ef0-991f-b0765b580c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:43.188881 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a2a8a-7f4f-4ef0-991f-b0765b580c6c admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:43.189169 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a2a8a-7f4f-4ef0-991f-b0765b580c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:43.189446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a2a8a-7f4f-4ef0-991f-b0765b580c6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:43.190141 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1a2a8a-7f4f-4ef0-991f-b0765b580c6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:43.192061 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1a2a8a-7f4f-4ef0-991f-b0765b580c6c admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:43.288351 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1429/2885] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 07:21:43 2022] POST /identity/v3/projects => generated 381 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 07:21:43.292516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d22604-e62b-46df-b4e7-9b05234bdaba admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:43.302928 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d22604-e62b-46df-b4e7-9b05234bdaba admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:43.357517 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d67f3-64ae-4395-a81c-6033d7e4a546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:43.358685 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d67f3-64ae-4395-a81c-6033d7e4a546 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:43.358970 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d67f3-64ae-4395-a81c-6033d7e4a546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:43.359247 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d67f3-64ae-4395-a81c-6033d7e4a546 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:43.359938 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978d67f3-64ae-4395-a81c-6033d7e4a546 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:43.361940 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978d67f3-64ae-4395-a81c-6033d7e4a546 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:43.446794 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1457/2886] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 07:21:43 2022] POST /identity/v3/users => generated 291 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 07:21:43.450827 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a2a8a-7f4f-4ef0-991f-b0765b580c6c admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:43.461579 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a2a8a-7f4f-4ef0-991f-b0765b580c6c admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:43.467227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f8fef-5fcb-498e-9ccc-05bac45966ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:43.467912 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f8fef-5fcb-498e-9ccc-05bac45966ba admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:43.468232 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f8fef-5fcb-498e-9ccc-05bac45966ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:43.468548 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f8fef-5fcb-498e-9ccc-05bac45966ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:43.469290 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56f8fef-5fcb-498e-9ccc-05bac45966ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:43.470866 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56f8fef-5fcb-498e-9ccc-05bac45966ba admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:43.481477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1430/2887] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:21:43.486178 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d67f3-64ae-4395-a81c-6033d7e4a546 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:43.496084 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d67f3-64ae-4395-a81c-6033d7e4a546 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:43.500935 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04960d5-99f4-4ec7-84f5-ae992af6e93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:43.501582 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04960d5-99f4-4ec7-84f5-ae992af6e93e admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:43.501867 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04960d5-99f4-4ec7-84f5-ae992af6e93e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:43.502145 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04960d5-99f4-4ec7-84f5-ae992af6e93e admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:43.502866 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04960d5-99f4-4ec7-84f5-ae992af6e93e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:43.504477 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04960d5-99f4-4ec7-84f5-ae992af6e93e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:43.514511 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1458/2888] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:21:43.518499 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f8fef-5fcb-498e-9ccc-05bac45966ba admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:43.528153 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f8fef-5fcb-498e-9ccc-05bac45966ba admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:43.533432 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774ca14-ac05-4b0b-b295-955999e6ff48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:43.534298 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774ca14-ac05-4b0b-b295-955999e6ff48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:43.534582 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774ca14-ac05-4b0b-b295-955999e6ff48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:43.534860 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774ca14-ac05-4b0b-b295-955999e6ff48 admin admin] PATH_INFO: `/v3/projects/be1d9278bb70494e967a08d0f3f5ee81/users/b78811ea37ca4715952314e44ec30a07/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:43.549900 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2774ca14-ac05-4b0b-b295-955999e6ff48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be1d9278bb70494e967a08d0f3f5ee81, user_id=b78811ea37ca4715952314e44ec30a07, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:43.552565 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2774ca14-ac05-4b0b-b295-955999e6ff48 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:43.649836 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1431/2889] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 07:21:43 2022] PUT /identity/v3/projects/be1d9278bb70494e967a08d0f3f5ee81/users/b78811ea37ca4715952314e44ec30a07/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:21:43.662043 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf979f-cb5f-4534-afa1-7b9a027fdf8f None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:43.662368 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf979f-cb5f-4534-afa1-7b9a027fdf8f None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:43.662651 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf979f-cb5f-4534-afa1-7b9a027fdf8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:43.690185 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-8bbf979f-cb5f-4534-afa1-7b9a027fdf8f None None] MFA Rules not processed for user `b78811ea37ca4715952314e44ec30a07`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:21:43.691393 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-8bbf979f-cb5f-4534-afa1-7b9a027fdf8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:43.755464 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1459/2890] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:21:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 07:21:43.760073 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774ca14-ac05-4b0b-b295-955999e6ff48 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:43.802220 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:21:43.821727 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774ca14-ac05-4b0b-b295-955999e6ff48 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:43.827125 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14179a-acc5-467d-b9be-66b4e94efbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:43.827802 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14179a-acc5-467d-b9be-66b4e94efbc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:43.828091 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14179a-acc5-467d-b9be-66b4e94efbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:43.828377 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14179a-acc5-467d-b9be-66b4e94efbc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:43.829089 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a14179a-acc5-467d-b9be-66b4e94efbc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:43.831065 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a14179a-acc5-467d-b9be-66b4e94efbc2 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:43.951913 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1432/2891] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 07:21:43 2022] POST /identity/v3/projects => generated 381 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 07:21:43.956010 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf979f-cb5f-4534-afa1-7b9a027fdf8f None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:43.967881 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf979f-cb5f-4534-afa1-7b9a027fdf8f None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:43.973021 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:21:44.023484 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ebecb4-b84a-409e-8e66-d7b8ed448c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:44.024155 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebecb4-b84a-409e-8e66-d7b8ed448c81 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:44.024468 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebecb4-b84a-409e-8e66-d7b8ed448c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:44.024751 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebecb4-b84a-409e-8e66-d7b8ed448c81 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:44.025485 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ebecb4-b84a-409e-8e66-d7b8ed448c81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:44.027383 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ebecb4-b84a-409e-8e66-d7b8ed448c81 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:44.128229 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1460/2892] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 07:21:43 2022] POST /identity/v3/users => generated 297 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 07:21:44.132036 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14179a-acc5-467d-b9be-66b4e94efbc2 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:44.142804 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14179a-acc5-467d-b9be-66b4e94efbc2 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:44.147297 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75816ffd-4d0f-44e5-825b-3ba19d10fc82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:44.147950 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75816ffd-4d0f-44e5-825b-3ba19d10fc82 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:44.148239 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75816ffd-4d0f-44e5-825b-3ba19d10fc82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:44.148663 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75816ffd-4d0f-44e5-825b-3ba19d10fc82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:44.149447 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75816ffd-4d0f-44e5-825b-3ba19d10fc82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:44.151025 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75816ffd-4d0f-44e5-825b-3ba19d10fc82 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:44.161841 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1433/2893] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:21:44.168628 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ebecb4-b84a-409e-8e66-d7b8ed448c81 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:44.179682 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ebecb4-b84a-409e-8e66-d7b8ed448c81 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:44.184493 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3210215-97fb-4439-9cfc-891bd887dc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:44.188653 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3210215-97fb-4439-9cfc-891bd887dc99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:44.188653 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3210215-97fb-4439-9cfc-891bd887dc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:44.188653 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3210215-97fb-4439-9cfc-891bd887dc99 admin admin] PATH_INFO: `/v3/projects/a8a2c5e068914b46893cc27d61e02ad9/users/d131e27071fd4135b93b6805e2d15c1b/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:44.201155 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3210215-97fb-4439-9cfc-891bd887dc99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8a2c5e068914b46893cc27d61e02ad9, user_id=d131e27071fd4135b93b6805e2d15c1b, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:44.203820 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3210215-97fb-4439-9cfc-891bd887dc99 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:44.275756 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1461/2894] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 07:21:44 2022] PUT /identity/v3/projects/a8a2c5e068914b46893cc27d61e02ad9/users/d131e27071fd4135b93b6805e2d15c1b/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:21:44.280210 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75816ffd-4d0f-44e5-825b-3ba19d10fc82 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:44.289923 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75816ffd-4d0f-44e5-825b-3ba19d10fc82 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:44.345835 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110f16c5-e7b3-4104-b29d-60f5212f0b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:44.346499 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110f16c5-e7b3-4104-b29d-60f5212f0b44 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:44.346782 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110f16c5-e7b3-4104-b29d-60f5212f0b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:44.347062 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110f16c5-e7b3-4104-b29d-60f5212f0b44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:44.347781 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110f16c5-e7b3-4104-b29d-60f5212f0b44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:44.349386 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110f16c5-e7b3-4104-b29d-60f5212f0b44 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:44.359686 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1434/2895] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:21:44.363892 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3210215-97fb-4439-9cfc-891bd887dc99 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:44.374504 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3210215-97fb-4439-9cfc-891bd887dc99 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:44.379597 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a302bc1-11fa-4f77-a2e6-2cef159ae8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:44.380430 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a302bc1-11fa-4f77-a2e6-2cef159ae8bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:44.380726 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a302bc1-11fa-4f77-a2e6-2cef159ae8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:44.381004 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a302bc1-11fa-4f77-a2e6-2cef159ae8bf admin admin] PATH_INFO: `/v3/projects/a8a2c5e068914b46893cc27d61e02ad9/users/d131e27071fd4135b93b6805e2d15c1b/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:44.382961 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a302bc1-11fa-4f77-a2e6-2cef159ae8bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8a2c5e068914b46893cc27d61e02ad9, user_id=d131e27071fd4135b93b6805e2d15c1b, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:44.385647 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a302bc1-11fa-4f77-a2e6-2cef159ae8bf admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:44.395879 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1462/2896] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 07:21:44 2022] PUT /identity/v3/projects/a8a2c5e068914b46893cc27d61e02ad9/users/d131e27071fd4135b93b6805e2d15c1b/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 07:21:44.406771 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f59dfe-8748-44a3-b150-051b97fc84f6 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:44.407063 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f59dfe-8748-44a3-b150-051b97fc84f6 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:44.407343 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f59dfe-8748-44a3-b150-051b97fc84f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:44.437332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-a0f59dfe-8748-44a3-b150-051b97fc84f6 None None] MFA Rules not processed for user `d131e27071fd4135b93b6805e2d15c1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:21:44.438527 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a0f59dfe-8748-44a3-b150-051b97fc84f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:44.506141 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1435/2897] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:21:44 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:21:44.517938 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a302bc1-11fa-4f77-a2e6-2cef159ae8bf admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:44.519683 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-5a302bc1-11fa-4f77-a2e6-2cef159ae8bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:44.577782 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a302bc1-11fa-4f77-a2e6-2cef159ae8bf admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:44.581517 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d12f14-97ed-4b9a-9f6c-5f66bd0feee2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:44.582171 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d12f14-97ed-4b9a-9f6c-5f66bd0feee2 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:44.582459 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d12f14-97ed-4b9a-9f6c-5f66bd0feee2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:44.582742 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d12f14-97ed-4b9a-9f6c-5f66bd0feee2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:44.592743 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d12f14-97ed-4b9a-9f6c-5f66bd0feee2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:44.594204 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d12f14-97ed-4b9a-9f6c-5f66bd0feee2 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:44.603672 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1463/2898] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:21:44 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:21:45.066956 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f59dfe-8748-44a3-b150-051b97fc84f6 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:45.105368 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:21:45.128688 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f59dfe-8748-44a3-b150-051b97fc84f6 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:45.133714 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf4f7c-9c31-4a68-aa61-a63c66328390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:45.134575 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf4f7c-9c31-4a68-aa61-a63c66328390 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:45.134856 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf4f7c-9c31-4a68-aa61-a63c66328390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:45.135153 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf4f7c-9c31-4a68-aa61-a63c66328390 admin admin] PATH_INFO: `/v3/users/b78811ea37ca4715952314e44ec30a07` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:45.136326 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaf4f7c-9c31-4a68-aa61-a63c66328390 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b78811ea37ca4715952314e44ec30a07)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:45.138243 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaf4f7c-9c31-4a68-aa61-a63c66328390 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:45.363098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-baaf4f7c-9c31-4a68-aa61-a63c66328390 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b78811ea37ca4715952314e44ec30a07'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:45.370000 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-baaf4f7c-9c31-4a68-aa61-a63c66328390 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b78811ea37ca4715952314e44ec30a07'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:45.382642 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-baaf4f7c-9c31-4a68-aa61-a63c66328390 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b78811ea37ca4715952314e44ec30a07'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:45.383329 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-baaf4f7c-9c31-4a68-aa61-a63c66328390 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b78811ea37ca4715952314e44ec30a07'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:45.383932 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-baaf4f7c-9c31-4a68-aa61-a63c66328390 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b78811ea37ca4715952314e44ec30a07'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:45.446786 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1436/2899] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 07:21:45 2022] DELETE /identity/v3/users/b78811ea37ca4715952314e44ec30a07 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:21:45.451363 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fe005-7c3e-40c0-8ce0-dade522edcdb None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:45.461426 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fe005-7c3e-40c0-8ce0-dade522edcdb None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:45.461426 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fe005-7c3e-40c0-8ce0-dade522edcdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:45.469683 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-661fe005-7c3e-40c0-8ce0-dade522edcdb None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:21:45.470891 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-661fe005-7c3e-40c0-8ce0-dade522edcdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:45.529317 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1464/2900] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:21:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:21:45.538958 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf4f7c-9c31-4a68-aa61-a63c66328390 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:45.540339 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-baaf4f7c-9c31-4a68-aa61-a63c66328390 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:45.597294 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf4f7c-9c31-4a68-aa61-a63c66328390 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:45.600836 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8114da19-55d0-4bd5-a70e-a0bf90dac3ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:45.601477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114da19-55d0-4bd5-a70e-a0bf90dac3ae service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:45.601764 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114da19-55d0-4bd5-a70e-a0bf90dac3ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:45.602069 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114da19-55d0-4bd5-a70e-a0bf90dac3ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:45.611549 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8114da19-55d0-4bd5-a70e-a0bf90dac3ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:45.612993 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8114da19-55d0-4bd5-a70e-a0bf90dac3ae service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:45.622319 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1437/2901] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:21:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:21:46.038808 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661fe005-7c3e-40c0-8ce0-dade522edcdb None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:46.040238 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-661fe005-7c3e-40c0-8ce0-dade522edcdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:46.048839 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:21:46.052789 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661fe005-7c3e-40c0-8ce0-dade522edcdb None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:46.058128 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41650ccd-43c1-43b6-8342-fcc0551453f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:46.058998 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41650ccd-43c1-43b6-8342-fcc0551453f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:46.059288 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41650ccd-43c1-43b6-8342-fcc0551453f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:46.059562 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41650ccd-43c1-43b6-8342-fcc0551453f7 admin admin] PATH_INFO: `/v3/projects/be1d9278bb70494e967a08d0f3f5ee81` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:46.060784 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41650ccd-43c1-43b6-8342-fcc0551453f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be1d9278bb70494e967a08d0f3f5ee81)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:46.062914 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41650ccd-43c1-43b6-8342-fcc0551453f7 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:46.173699 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-41650ccd-43c1-43b6-8342-fcc0551453f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be1d9278bb70494e967a08d0f3f5ee81', 'request_id': 'req-41650ccd-43c1-43b6-8342-fcc0551453f7'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:46.179961 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-41650ccd-43c1-43b6-8342-fcc0551453f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be1d9278bb70494e967a08d0f3f5ee81', 'request_id': 'req-41650ccd-43c1-43b6-8342-fcc0551453f7'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:46.243692 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-41650ccd-43c1-43b6-8342-fcc0551453f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be1d9278bb70494e967a08d0f3f5ee81', 'request_id': 'req-41650ccd-43c1-43b6-8342-fcc0551453f7'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:46.255101 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-41650ccd-43c1-43b6-8342-fcc0551453f7 admin admin] The token cache is being invalidate because project be1d9278bb70494e967a08d0f3f5ee81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 07:21:46.255650 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-41650ccd-43c1-43b6-8342-fcc0551453f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:46.256859 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-41650ccd-43c1-43b6-8342-fcc0551453f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:46.258460 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1465/2902] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 07:21:46 2022] DELETE /identity/v3/projects/be1d9278bb70494e967a08d0f3f5ee81 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:21:46.262496 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8114da19-55d0-4bd5-a70e-a0bf90dac3ae service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:46.272300 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8114da19-55d0-4bd5-a70e-a0bf90dac3ae service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:46.324708 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:21:46.330860 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f50a36-581f-49e1-a336-5a4d0f2d8ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:46.331728 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f50a36-581f-49e1-a336-5a4d0f2d8ab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:46.332013 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f50a36-581f-49e1-a336-5a4d0f2d8ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:46.332288 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f50a36-581f-49e1-a336-5a4d0f2d8ab6 admin admin] PATH_INFO: `/v3/users/d131e27071fd4135b93b6805e2d15c1b` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:46.333495 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f50a36-581f-49e1-a336-5a4d0f2d8ab6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d131e27071fd4135b93b6805e2d15c1b)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:46.335557 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f50a36-581f-49e1-a336-5a4d0f2d8ab6 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:46.517451 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-20f50a36-581f-49e1-a336-5a4d0f2d8ab6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd131e27071fd4135b93b6805e2d15c1b'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:46.524146 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-20f50a36-581f-49e1-a336-5a4d0f2d8ab6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd131e27071fd4135b93b6805e2d15c1b'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:46.538826 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-20f50a36-581f-49e1-a336-5a4d0f2d8ab6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd131e27071fd4135b93b6805e2d15c1b'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:46.539546 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-20f50a36-581f-49e1-a336-5a4d0f2d8ab6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd131e27071fd4135b93b6805e2d15c1b'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:46.540159 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-20f50a36-581f-49e1-a336-5a4d0f2d8ab6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd131e27071fd4135b93b6805e2d15c1b'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:46.612645 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1438/2903] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 07:21:46 2022] DELETE /identity/v3/users/d131e27071fd4135b93b6805e2d15c1b => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:21:46.993729 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41650ccd-43c1-43b6-8342-fcc0551453f7 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:46.995201 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-41650ccd-43c1-43b6-8342-fcc0551453f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:47.009970 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41650ccd-43c1-43b6-8342-fcc0551453f7 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:47.068618 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4cf26-8505-4b02-a490-5cfe5edf96ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:47.069493 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4cf26-8505-4b02-a490-5cfe5edf96ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:47.069783 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4cf26-8505-4b02-a490-5cfe5edf96ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:47.070060 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4cf26-8505-4b02-a490-5cfe5edf96ab admin admin] PATH_INFO: `/v3/projects/a8a2c5e068914b46893cc27d61e02ad9` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:47.071511 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc4cf26-8505-4b02-a490-5cfe5edf96ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8a2c5e068914b46893cc27d61e02ad9)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:47.073576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc4cf26-8505-4b02-a490-5cfe5edf96ab admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:47.196338 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0bc4cf26-8505-4b02-a490-5cfe5edf96ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8a2c5e068914b46893cc27d61e02ad9', 'request_id': 'req-0bc4cf26-8505-4b02-a490-5cfe5edf96ab'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:47.202828 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0bc4cf26-8505-4b02-a490-5cfe5edf96ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8a2c5e068914b46893cc27d61e02ad9', 'request_id': 'req-0bc4cf26-8505-4b02-a490-5cfe5edf96ab'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:47.289478 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0bc4cf26-8505-4b02-a490-5cfe5edf96ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8a2c5e068914b46893cc27d61e02ad9', 'request_id': 'req-0bc4cf26-8505-4b02-a490-5cfe5edf96ab'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:47.301327 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0bc4cf26-8505-4b02-a490-5cfe5edf96ab admin admin] The token cache is being invalidate because project a8a2c5e068914b46893cc27d61e02ad9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 07:21:47.301617 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0bc4cf26-8505-4b02-a490-5cfe5edf96ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:47.302249 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0bc4cf26-8505-4b02-a490-5cfe5edf96ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:21:47.303787 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1466/2904] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 07:21:46 2022] DELETE /identity/v3/projects/a8a2c5e068914b46893cc27d61e02ad9 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:21:47.308829 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb97c4-79a7-4c1c-b5d7-08a1b71e3c2f None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:47.309313 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb97c4-79a7-4c1c-b5d7-08a1b71e3c2f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:47.309418 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb97c4-79a7-4c1c-b5d7-08a1b71e3c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:47.328778 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-87eb97c4-79a7-4c1c-b5d7-08a1b71e3c2f None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:21:47.329991 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-87eb97c4-79a7-4c1c-b5d7-08a1b71e3c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:47.387828 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1439/2905] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:21:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:21:47.400939 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867989ad-9cb3-4a52-9f8f-72aec1c58fd6 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:47.401227 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867989ad-9cb3-4a52-9f8f-72aec1c58fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:47.401505 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867989ad-9cb3-4a52-9f8f-72aec1c58fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:47.420767 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-867989ad-9cb3-4a52-9f8f-72aec1c58fd6 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:21:47.421969 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-867989ad-9cb3-4a52-9f8f-72aec1c58fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:47.429689 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1467/2906] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:21:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:21:47.433812 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb97c4-79a7-4c1c-b5d7-08a1b71e3c2f None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:47.443732 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb97c4-79a7-4c1c-b5d7-08a1b71e3c2f None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:47.448554 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732076a-0e2a-4a80-9a0f-d6b292ce4d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:47.449203 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732076a-0e2a-4a80-9a0f-d6b292ce4d16 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:47.449490 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732076a-0e2a-4a80-9a0f-d6b292ce4d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:47.449768 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732076a-0e2a-4a80-9a0f-d6b292ce4d16 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:47.450495 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b732076a-0e2a-4a80-9a0f-d6b292ce4d16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 07:21:47.450836 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b732076a-0e2a-4a80-9a0f-d6b292ce4d16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:47.453001 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b732076a-0e2a-4a80-9a0f-d6b292ce4d16 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:47.466069 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1440/2907] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 07:21:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 07:21:47.469375 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867989ad-9cb3-4a52-9f8f-72aec1c58fd6 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:47.478577 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867989ad-9cb3-4a52-9f8f-72aec1c58fd6 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:47.483342 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed1992f-d813-44c2-8777-c8fb7924286e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:47.484008 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1992f-d813-44c2-8777-c8fb7924286e admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:47.484301 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1992f-d813-44c2-8777-c8fb7924286e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:47.484622 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1992f-d813-44c2-8777-c8fb7924286e admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:47.485329 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed1992f-d813-44c2-8777-c8fb7924286e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:47.487230 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed1992f-d813-44c2-8777-c8fb7924286e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:47.495327 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:21:47.574527 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1468/2908] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 07:21:47 2022] POST /identity/v3/projects => generated 371 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 07:21:47.579130 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732076a-0e2a-4a80-9a0f-d6b292ce4d16 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:47.589489 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732076a-0e2a-4a80-9a0f-d6b292ce4d16 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:47.646417 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688abc72-f8f3-4339-8df0-67ff2b5fafbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:47.647084 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688abc72-f8f3-4339-8df0-67ff2b5fafbf admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:47.647369 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688abc72-f8f3-4339-8df0-67ff2b5fafbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:47.647646 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688abc72-f8f3-4339-8df0-67ff2b5fafbf admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:47.648326 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688abc72-f8f3-4339-8df0-67ff2b5fafbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:47.650308 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688abc72-f8f3-4339-8df0-67ff2b5fafbf admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:47.659438 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:21:47.781918 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1441/2909] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 07:21:47 2022] POST /identity/v3/users => generated 286 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 07:21:47.786096 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed1992f-d813-44c2-8777-c8fb7924286e admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:47.796676 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed1992f-d813-44c2-8777-c8fb7924286e admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:47.801170 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c21370-61a8-47e3-ba56-588ad0bcd832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:47.801827 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c21370-61a8-47e3-ba56-588ad0bcd832 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:47.802113 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c21370-61a8-47e3-ba56-588ad0bcd832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:47.802478 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c21370-61a8-47e3-ba56-588ad0bcd832 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:47.803110 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c21370-61a8-47e3-ba56-588ad0bcd832 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:47.804761 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c21370-61a8-47e3-ba56-588ad0bcd832 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:47.815204 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1469/2910] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:21:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:21:47.819079 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688abc72-f8f3-4339-8df0-67ff2b5fafbf admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:47.830281 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688abc72-f8f3-4339-8df0-67ff2b5fafbf admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:47.834713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d6be6-90ed-4b12-b031-c91550f30ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:47.835365 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d6be6-90ed-4b12-b031-c91550f30ac1 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:47.835655 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d6be6-90ed-4b12-b031-c91550f30ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:47.835925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d6be6-90ed-4b12-b031-c91550f30ac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:47.836686 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121d6be6-90ed-4b12-b031-c91550f30ac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:47.838256 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121d6be6-90ed-4b12-b031-c91550f30ac1 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:47.848984 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1442/2911] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:21:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:21:47.852954 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c21370-61a8-47e3-ba56-588ad0bcd832 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:47.863017 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c21370-61a8-47e3-ba56-588ad0bcd832 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:47.867439 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a61ca2-c3c0-48e8-9570-96b01534237b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:47.868312 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a61ca2-c3c0-48e8-9570-96b01534237b admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:47.868650 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a61ca2-c3c0-48e8-9570-96b01534237b admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:47.868935 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a61ca2-c3c0-48e8-9570-96b01534237b admin admin] PATH_INFO: `/v3/projects/743167026c114052b754cf0b528f0e6a/users/a95efecd8d6f4d2bb7626c88cb2bc7e4/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:47.884109 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a61ca2-c3c0-48e8-9570-96b01534237b admin admin] RBAC: Authorizing `identity:create_grant(project_id=743167026c114052b754cf0b528f0e6a, user_id=a95efecd8d6f4d2bb7626c88cb2bc7e4, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:47.886793 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a61ca2-c3c0-48e8-9570-96b01534237b admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:47.987106 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1470/2912] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 07:21:47 2022] PUT /identity/v3/projects/743167026c114052b754cf0b528f0e6a/users/a95efecd8d6f4d2bb7626c88cb2bc7e4/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:21:47.995772 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494d3b8-c313-41f0-b9ae-6725cbf23096 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:47.996062 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494d3b8-c313-41f0-b9ae-6725cbf23096 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:47.996345 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494d3b8-c313-41f0-b9ae-6725cbf23096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:48.013582 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-4494d3b8-c313-41f0-b9ae-6725cbf23096 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:21:48.015382 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4494d3b8-c313-41f0-b9ae-6725cbf23096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:48.072403 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1443/2913] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:21:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:21:48.082685 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a61ca2-c3c0-48e8-9570-96b01534237b admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:48.084408 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-25a61ca2-c3c0-48e8-9570-96b01534237b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:48.142254 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a61ca2-c3c0-48e8-9570-96b01534237b admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:48.145933 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3129940-ac18-4a85-99ff-842494550e31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:48.146576 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3129940-ac18-4a85-99ff-842494550e31 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:48.146868 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3129940-ac18-4a85-99ff-842494550e31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:48.147151 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3129940-ac18-4a85-99ff-842494550e31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:48.157284 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3129940-ac18-4a85-99ff-842494550e31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:48.158714 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3129940-ac18-4a85-99ff-842494550e31 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:48.168581 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1471/2914] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:21:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:21:49.360053 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce766443-4c58-4c64-96c7-520c93b926ef None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:49.360585 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce766443-4c58-4c64-96c7-520c93b926ef None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:49.360771 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce766443-4c58-4c64-96c7-520c93b926ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:49.386930 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-ce766443-4c58-4c64-96c7-520c93b926ef None None] MFA Rules not processed for user `a95efecd8d6f4d2bb7626c88cb2bc7e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:21:49.388150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ce766443-4c58-4c64-96c7-520c93b926ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:49.454553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1444/2915] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:21:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:21:49.459042 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3129940-ac18-4a85-99ff-842494550e31 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:49.469466 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3129940-ac18-4a85-99ff-842494550e31 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:49.473897 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cff1c0-0c82-4e9c-8dc6-cdc8b4930b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:49.474537 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cff1c0-0c82-4e9c-8dc6-cdc8b4930b8e admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:49.474822 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cff1c0-0c82-4e9c-8dc6-cdc8b4930b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:49.475099 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cff1c0-0c82-4e9c-8dc6-cdc8b4930b8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:49.475789 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cff1c0-0c82-4e9c-8dc6-cdc8b4930b8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:49.477706 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cff1c0-0c82-4e9c-8dc6-cdc8b4930b8e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:49.564771 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1472/2916] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 07:21:49 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 07:21:49.568654 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce766443-4c58-4c64-96c7-520c93b926ef None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:49.584423 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce766443-4c58-4c64-96c7-520c93b926ef None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:49.634138 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec10303-1e15-4749-92e0-c428c865a437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:49.634802 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec10303-1e15-4749-92e0-c428c865a437 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:49.635088 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec10303-1e15-4749-92e0-c428c865a437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:49.635364 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec10303-1e15-4749-92e0-c428c865a437 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:49.636041 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec10303-1e15-4749-92e0-c428c865a437 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:49.638021 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec10303-1e15-4749-92e0-c428c865a437 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:49.719303 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1445/2917] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 07:21:49 2022] POST /identity/v3/users => generated 293 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 07:21:49.723546 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cff1c0-0c82-4e9c-8dc6-cdc8b4930b8e admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:49.733967 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cff1c0-0c82-4e9c-8dc6-cdc8b4930b8e admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:49.738582 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18672654-b6d3-4e43-a437-7dd66b33e210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:49.739233 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18672654-b6d3-4e43-a437-7dd66b33e210 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:49.739517 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18672654-b6d3-4e43-a437-7dd66b33e210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:49.739824 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18672654-b6d3-4e43-a437-7dd66b33e210 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:49.740581 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18672654-b6d3-4e43-a437-7dd66b33e210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:49.742172 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18672654-b6d3-4e43-a437-7dd66b33e210 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:49.752625 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1473/2918] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:21:49.756676 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec10303-1e15-4749-92e0-c428c865a437 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:49.761801 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:21:49.767835 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec10303-1e15-4749-92e0-c428c865a437 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:49.774752 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cabfc40-1372-438f-adf8-e7f20f53d764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:49.774752 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabfc40-1372-438f-adf8-e7f20f53d764 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:49.774752 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabfc40-1372-438f-adf8-e7f20f53d764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:49.774752 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabfc40-1372-438f-adf8-e7f20f53d764 admin admin] PATH_INFO: `/v3/projects/5cdc86523e0947a0b7ebafa82b8ac765/users/a42477362e884cdb9f95b4177b2eb62c/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:49.788967 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cabfc40-1372-438f-adf8-e7f20f53d764 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cdc86523e0947a0b7ebafa82b8ac765, user_id=a42477362e884cdb9f95b4177b2eb62c, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:49.791594 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cabfc40-1372-438f-adf8-e7f20f53d764 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:49.852482 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1446/2919] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 07:21:49 2022] PUT /identity/v3/projects/5cdc86523e0947a0b7ebafa82b8ac765/users/a42477362e884cdb9f95b4177b2eb62c/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 07:21:49.856390 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18672654-b6d3-4e43-a437-7dd66b33e210 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:49.866802 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18672654-b6d3-4e43-a437-7dd66b33e210 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:49.922161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edcef7d-9793-4fbd-8ee8-fd2594c564ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:49.922835 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edcef7d-9793-4fbd-8ee8-fd2594c564ab admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:49.923122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edcef7d-9793-4fbd-8ee8-fd2594c564ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:49.923399 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edcef7d-9793-4fbd-8ee8-fd2594c564ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:49.924159 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edcef7d-9793-4fbd-8ee8-fd2594c564ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:49.925881 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edcef7d-9793-4fbd-8ee8-fd2594c564ab admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:49.935643 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1474/2920] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 07:21:49.939738 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cabfc40-1372-438f-adf8-e7f20f53d764 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:49.950271 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cabfc40-1372-438f-adf8-e7f20f53d764 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:49.955549 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c611408-62fa-4514-9507-39575ebf58d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:49.956441 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c611408-62fa-4514-9507-39575ebf58d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:49.956734 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c611408-62fa-4514-9507-39575ebf58d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:49.957040 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c611408-62fa-4514-9507-39575ebf58d4 admin admin] PATH_INFO: `/v3/projects/5cdc86523e0947a0b7ebafa82b8ac765/users/a42477362e884cdb9f95b4177b2eb62c/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:49.959016 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c611408-62fa-4514-9507-39575ebf58d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cdc86523e0947a0b7ebafa82b8ac765, user_id=a42477362e884cdb9f95b4177b2eb62c, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:49.961700 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c611408-62fa-4514-9507-39575ebf58d4 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:49.972853 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1447/2921] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 07:21:49 2022] PUT /identity/v3/projects/5cdc86523e0947a0b7ebafa82b8ac765/users/a42477362e884cdb9f95b4177b2eb62c/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:21:51.156916 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9413522-4843-4365-b7e7-a6c7dd8966a7 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:51.157414 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9413522-4843-4365-b7e7-a6c7dd8966a7 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:51.157604 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9413522-4843-4365-b7e7-a6c7dd8966a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:51.184870 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-e9413522-4843-4365-b7e7-a6c7dd8966a7 None None] MFA Rules not processed for user `a42477362e884cdb9f95b4177b2eb62c`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:21:51.186278 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e9413522-4843-4365-b7e7-a6c7dd8966a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:51.194871 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:21:51.256733 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1475/2922] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:21:51 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:21:51.268924 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c611408-62fa-4514-9507-39575ebf58d4 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:51.270672 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8c611408-62fa-4514-9507-39575ebf58d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:51.327777 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c611408-62fa-4514-9507-39575ebf58d4 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:51.331224 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06eb63-1120-40ef-bb57-ee9fe335a791 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:51.331855 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06eb63-1120-40ef-bb57-ee9fe335a791 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:51.332137 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06eb63-1120-40ef-bb57-ee9fe335a791 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:51.332493 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06eb63-1120-40ef-bb57-ee9fe335a791 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:51.386215 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06eb63-1120-40ef-bb57-ee9fe335a791 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:51.387629 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06eb63-1120-40ef-bb57-ee9fe335a791 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:51.397091 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1448/2923] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:21:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:21:53.241856 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9413522-4843-4365-b7e7-a6c7dd8966a7 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:53.243310 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e9413522-4843-4365-b7e7-a6c7dd8966a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:53.255574 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9413522-4843-4365-b7e7-a6c7dd8966a7 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:53.259406 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020b300f-8b36-4307-a63f-ddd70231ee14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:53.261504 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b300f-8b36-4307-a63f-ddd70231ee14 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:53.261504 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b300f-8b36-4307-a63f-ddd70231ee14 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:53.261504 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b300f-8b36-4307-a63f-ddd70231ee14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:53.270261 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020b300f-8b36-4307-a63f-ddd70231ee14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:53.271690 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020b300f-8b36-4307-a63f-ddd70231ee14 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:53.282135 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1476/2924] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:21:53 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:21:59.162237 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42705bcd-5165-492d-a44e-acf0b57de4e4 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:59.162780 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42705bcd-5165-492d-a44e-acf0b57de4e4 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:59.162780 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42705bcd-5165-492d-a44e-acf0b57de4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:59.181094 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-42705bcd-5165-492d-a44e-acf0b57de4e4 None None] MFA Rules not processed for user `a3560534427544cdabfd0ba702a0a165`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:21:59.182303 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-42705bcd-5165-492d-a44e-acf0b57de4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:59.244320 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1449/2925] 10.222.0.17 () {58 vars in 1015 bytes} [Mon Jul 18 07:21:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:21:59.268992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020b300f-8b36-4307-a63f-ddd70231ee14 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:21:59.270430 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-020b300f-8b36-4307-a63f-ddd70231ee14 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:21:59.286686 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020b300f-8b36-4307-a63f-ddd70231ee14 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:21:59.291906 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c7a98-d8bf-4985-8ab6-e28e1a0d7217 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3560534427544cdabfd0ba702a0a165', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:21:59.292583 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c7a98-d8bf-4985-8ab6-e28e1a0d7217 service placement] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:21:59.292878 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c7a98-d8bf-4985-8ab6-e28e1a0d7217 service placement] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:21:59.293171 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c7a98-d8bf-4985-8ab6-e28e1a0d7217 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:21:59.302662 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c7a98-d8bf-4985-8ab6-e28e1a0d7217 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:21:59.304241 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c7a98-d8bf-4985-8ab6-e28e1a0d7217 service placement] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:21:59.318335 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1477/2926] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:21:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:22:09.133826 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75974e69-6aad-4f3c-b629-2eca2678f9a0 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:09.134279 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75974e69-6aad-4f3c-b629-2eca2678f9a0 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:09.134424 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75974e69-6aad-4f3c-b629-2eca2678f9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:09.153746 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-75974e69-6aad-4f3c-b629-2eca2678f9a0 None None] MFA Rules not processed for user `a3560534427544cdabfd0ba702a0a165`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:22:09.154965 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-75974e69-6aad-4f3c-b629-2eca2678f9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:22:09.164220 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1450/2927] 10.222.0.40 () {58 vars in 1058 bytes} [Mon Jul 18 07:22:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:22:09.170387 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c7a98-d8bf-4985-8ab6-e28e1a0d7217 service placement] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:22:09.181523 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c7a98-d8bf-4985-8ab6-e28e1a0d7217 service placement] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:22:09.186731 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb6b4c-0d0d-4dc8-845c-ff899790adf2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3560534427544cdabfd0ba702a0a165', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:22:09.188021 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb6b4c-0d0d-4dc8-845c-ff899790adf2 service placement] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:09.188311 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb6b4c-0d0d-4dc8-845c-ff899790adf2 service placement] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:09.188630 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb6b4c-0d0d-4dc8-845c-ff899790adf2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:09.190459 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-b0bb6b4c-0d0d-4dc8-845c-ff899790adf2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:22:09.202556 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bb6b4c-0d0d-4dc8-845c-ff899790adf2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:22:09.207505 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bb6b4c-0d0d-4dc8-845c-ff899790adf2 service placement] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:22:09.214147 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1478/2928] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:22:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:22:09.906514 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75974e69-6aad-4f3c-b629-2eca2678f9a0 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:22:09.918212 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:22:09.970191 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75974e69-6aad-4f3c-b629-2eca2678f9a0 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:22:09.975143 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8adcc2-3935-48d1-bce9-2fb5a3b088eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:22:09.976008 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8adcc2-3935-48d1-bce9-2fb5a3b088eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:09.976296 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8adcc2-3935-48d1-bce9-2fb5a3b088eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:09.976602 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8adcc2-3935-48d1-bce9-2fb5a3b088eb admin admin] PATH_INFO: `/v3/users/a95efecd8d6f4d2bb7626c88cb2bc7e4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:09.977787 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8adcc2-3935-48d1-bce9-2fb5a3b088eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a95efecd8d6f4d2bb7626c88cb2bc7e4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:22:09.982267 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8adcc2-3935-48d1-bce9-2fb5a3b088eb admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:22:10.143077 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-4a8adcc2-3935-48d1-bce9-2fb5a3b088eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a95efecd8d6f4d2bb7626c88cb2bc7e4'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:10.150026 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-4a8adcc2-3935-48d1-bce9-2fb5a3b088eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a95efecd8d6f4d2bb7626c88cb2bc7e4'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:10.162798 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-4a8adcc2-3935-48d1-bce9-2fb5a3b088eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a95efecd8d6f4d2bb7626c88cb2bc7e4'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:10.163501 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-4a8adcc2-3935-48d1-bce9-2fb5a3b088eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a95efecd8d6f4d2bb7626c88cb2bc7e4'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:10.164145 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-4a8adcc2-3935-48d1-bce9-2fb5a3b088eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a95efecd8d6f4d2bb7626c88cb2bc7e4'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:10.275441 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1451/2929] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 07:22:09 2022] DELETE /identity/v3/users/a95efecd8d6f4d2bb7626c88cb2bc7e4 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:22:10.550768 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb6b4c-0d0d-4dc8-845c-ff899790adf2 service placement] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:22:10.561580 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb6b4c-0d0d-4dc8-845c-ff899790adf2 service placement] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:22:10.623094 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc40c5a-f5c6-4e32-b558-5914eba34ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:22:10.623954 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc40c5a-f5c6-4e32-b558-5914eba34ae8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:10.624242 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc40c5a-f5c6-4e32-b558-5914eba34ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:10.625118 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc40c5a-f5c6-4e32-b558-5914eba34ae8 admin admin] PATH_INFO: `/v3/projects/743167026c114052b754cf0b528f0e6a` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:10.626295 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc40c5a-f5c6-4e32-b558-5914eba34ae8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=743167026c114052b754cf0b528f0e6a)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:22:10.628192 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc40c5a-f5c6-4e32-b558-5914eba34ae8 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:22:10.736122 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6cc40c5a-f5c6-4e32-b558-5914eba34ae8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '743167026c114052b754cf0b528f0e6a', 'request_id': 'req-6cc40c5a-f5c6-4e32-b558-5914eba34ae8'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:10.742627 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6cc40c5a-f5c6-4e32-b558-5914eba34ae8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '743167026c114052b754cf0b528f0e6a', 'request_id': 'req-6cc40c5a-f5c6-4e32-b558-5914eba34ae8'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:10.803030 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6cc40c5a-f5c6-4e32-b558-5914eba34ae8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '743167026c114052b754cf0b528f0e6a', 'request_id': 'req-6cc40c5a-f5c6-4e32-b558-5914eba34ae8'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:10.807990 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:22:10.814516 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6cc40c5a-f5c6-4e32-b558-5914eba34ae8 admin admin] The token cache is being invalidate because project 743167026c114052b754cf0b528f0e6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 07:22:10.814792 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6cc40c5a-f5c6-4e32-b558-5914eba34ae8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:10.824106 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6cc40c5a-f5c6-4e32-b558-5914eba34ae8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:10.824106 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1479/2930] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 07:22:10 2022] DELETE /identity/v3/projects/743167026c114052b754cf0b528f0e6a => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:22:10.824311 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8adcc2-3935-48d1-bce9-2fb5a3b088eb admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:22:10.824311 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4a8adcc2-3935-48d1-bce9-2fb5a3b088eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:22:10.835577 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8adcc2-3935-48d1-bce9-2fb5a3b088eb admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:22:10.891782 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeedf6e-38b5-4be7-ae89-514b245afd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:22:10.892671 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeedf6e-38b5-4be7-ae89-514b245afd34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:10.892949 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeedf6e-38b5-4be7-ae89-514b245afd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:10.893220 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeedf6e-38b5-4be7-ae89-514b245afd34 admin admin] PATH_INFO: `/v3/users/a42477362e884cdb9f95b4177b2eb62c` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:10.894440 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeedf6e-38b5-4be7-ae89-514b245afd34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a42477362e884cdb9f95b4177b2eb62c)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:22:10.896314 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeedf6e-38b5-4be7-ae89-514b245afd34 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:22:11.033422 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:22:11.047236 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-cbeedf6e-38b5-4be7-ae89-514b245afd34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a42477362e884cdb9f95b4177b2eb62c'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:11.054313 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-cbeedf6e-38b5-4be7-ae89-514b245afd34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a42477362e884cdb9f95b4177b2eb62c'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:11.065989 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-cbeedf6e-38b5-4be7-ae89-514b245afd34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a42477362e884cdb9f95b4177b2eb62c'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:11.066667 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-cbeedf6e-38b5-4be7-ae89-514b245afd34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a42477362e884cdb9f95b4177b2eb62c'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:11.067279 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-cbeedf6e-38b5-4be7-ae89-514b245afd34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a42477362e884cdb9f95b4177b2eb62c'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:11.126449 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1452/2931] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 07:22:10 2022] DELETE /identity/v3/users/a42477362e884cdb9f95b4177b2eb62c => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:22:11.366678 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc40c5a-f5c6-4e32-b558-5914eba34ae8 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:22:11.368203 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-6cc40c5a-f5c6-4e32-b558-5914eba34ae8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:22:11.381298 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc40c5a-f5c6-4e32-b558-5914eba34ae8 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:22:11.440867 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efc4576-f5a4-4d7b-aa88-a2532194f4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:22:11.441728 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc4576-f5a4-4d7b-aa88-a2532194f4c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:11.442028 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc4576-f5a4-4d7b-aa88-a2532194f4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:11.442446 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc4576-f5a4-4d7b-aa88-a2532194f4c0 admin admin] PATH_INFO: `/v3/projects/5cdc86523e0947a0b7ebafa82b8ac765` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:11.443599 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efc4576-f5a4-4d7b-aa88-a2532194f4c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cdc86523e0947a0b7ebafa82b8ac765)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:22:11.445818 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efc4576-f5a4-4d7b-aa88-a2532194f4c0 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:22:11.655822 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1efc4576-f5a4-4d7b-aa88-a2532194f4c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cdc86523e0947a0b7ebafa82b8ac765', 'request_id': 'req-1efc4576-f5a4-4d7b-aa88-a2532194f4c0'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:11.662912 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1efc4576-f5a4-4d7b-aa88-a2532194f4c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cdc86523e0947a0b7ebafa82b8ac765', 'request_id': 'req-1efc4576-f5a4-4d7b-aa88-a2532194f4c0'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:11.723465 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1efc4576-f5a4-4d7b-aa88-a2532194f4c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cdc86523e0947a0b7ebafa82b8ac765', 'request_id': 'req-1efc4576-f5a4-4d7b-aa88-a2532194f4c0'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:11.734659 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1efc4576-f5a4-4d7b-aa88-a2532194f4c0 admin admin] The token cache is being invalidate because project 5cdc86523e0947a0b7ebafa82b8ac765 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 07:22:11.734954 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1efc4576-f5a4-4d7b-aa88-a2532194f4c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:11.735624 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1efc4576-f5a4-4d7b-aa88-a2532194f4c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:11.737201 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1480/2932] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 07:22:11 2022] DELETE /identity/v3/projects/5cdc86523e0947a0b7ebafa82b8ac765 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:22:11.742060 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae96cef2-be72-4225-a75c-1d9fbf6f871f None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:11.742531 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae96cef2-be72-4225-a75c-1d9fbf6f871f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:11.742670 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae96cef2-be72-4225-a75c-1d9fbf6f871f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:11.760705 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-ae96cef2-be72-4225-a75c-1d9fbf6f871f None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:22:11.761932 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ae96cef2-be72-4225-a75c-1d9fbf6f871f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:22:11.821481 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1453/2933] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:22:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:22:11.830990 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c881c0-9d79-4677-a793-4a7e232ae47b None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:11.832535 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c881c0-9d79-4677-a793-4a7e232ae47b None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:11.832535 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c881c0-9d79-4677-a793-4a7e232ae47b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:11.849949 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-71c881c0-9d79-4677-a793-4a7e232ae47b None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:22:11.851079 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-71c881c0-9d79-4677-a793-4a7e232ae47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:22:11.859037 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1481/2934] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:22:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:22:11.863280 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae96cef2-be72-4225-a75c-1d9fbf6f871f None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:22:11.873113 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae96cef2-be72-4225-a75c-1d9fbf6f871f None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:22:11.878385 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4c3a7-af37-4070-9f3c-e51a30d03687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:22:11.879036 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4c3a7-af37-4070-9f3c-e51a30d03687 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:11.879320 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4c3a7-af37-4070-9f3c-e51a30d03687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:11.879598 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4c3a7-af37-4070-9f3c-e51a30d03687 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:11.880317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4c3a7-af37-4070-9f3c-e51a30d03687 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 07:22:11.880652 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4c3a7-af37-4070-9f3c-e51a30d03687 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:22:11.882220 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4c3a7-af37-4070-9f3c-e51a30d03687 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:22:11.894640 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1454/2935] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 07:22:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 07:22:11.899022 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c881c0-9d79-4677-a793-4a7e232ae47b None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:22:11.908314 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c881c0-9d79-4677-a793-4a7e232ae47b None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:22:11.913148 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58da98-cfeb-49f2-97aa-08c073f92a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:22:11.913817 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58da98-cfeb-49f2-97aa-08c073f92a56 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:11.914096 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58da98-cfeb-49f2-97aa-08c073f92a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:11.914377 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58da98-cfeb-49f2-97aa-08c073f92a56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:11.915055 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58da98-cfeb-49f2-97aa-08c073f92a56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:22:11.916999 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58da98-cfeb-49f2-97aa-08c073f92a56 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:22:11.996174 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1482/2936] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 07:22:11 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 07:22:12.001045 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4c3a7-af37-4070-9f3c-e51a30d03687 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:22:12.010956 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4c3a7-af37-4070-9f3c-e51a30d03687 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:22:12.066175 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b61002c-8c27-408d-885e-8b2d3ec43232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:22:12.066841 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b61002c-8c27-408d-885e-8b2d3ec43232 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:12.067126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b61002c-8c27-408d-885e-8b2d3ec43232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:12.067408 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b61002c-8c27-408d-885e-8b2d3ec43232 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:12.068091 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b61002c-8c27-408d-885e-8b2d3ec43232 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:22:12.070287 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b61002c-8c27-408d-885e-8b2d3ec43232 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:22:12.144366 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1455/2937] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 07:22:12 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 07:22:12.148601 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58da98-cfeb-49f2-97aa-08c073f92a56 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:22:12.159125 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58da98-cfeb-49f2-97aa-08c073f92a56 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:22:12.163868 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce248f3-f29c-46d9-bc75-0ae1f960b94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:22:12.164560 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce248f3-f29c-46d9-bc75-0ae1f960b94c admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:12.164848 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce248f3-f29c-46d9-bc75-0ae1f960b94c admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:12.165261 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce248f3-f29c-46d9-bc75-0ae1f960b94c admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:12.165989 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce248f3-f29c-46d9-bc75-0ae1f960b94c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:22:12.167573 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce248f3-f29c-46d9-bc75-0ae1f960b94c admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:22:12.177967 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1483/2938] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:22:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:22:12.181823 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b61002c-8c27-408d-885e-8b2d3ec43232 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:22:12.192602 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b61002c-8c27-408d-885e-8b2d3ec43232 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:22:12.197385 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb014c-1faf-41a7-8f72-9f00efbec623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:22:12.198028 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb014c-1faf-41a7-8f72-9f00efbec623 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:12.198311 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb014c-1faf-41a7-8f72-9f00efbec623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:12.198586 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb014c-1faf-41a7-8f72-9f00efbec623 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:12.199297 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb014c-1faf-41a7-8f72-9f00efbec623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:22:12.200890 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb014c-1faf-41a7-8f72-9f00efbec623 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:22:12.211024 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1456/2939] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:22:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:22:12.214912 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce248f3-f29c-46d9-bc75-0ae1f960b94c admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:22:12.224929 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce248f3-f29c-46d9-bc75-0ae1f960b94c admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:22:12.229458 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb5e72a-ac1b-4b01-b543-601e6936cf6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:22:12.230321 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5e72a-ac1b-4b01-b543-601e6936cf6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:12.230607 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5e72a-ac1b-4b01-b543-601e6936cf6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:12.230887 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5e72a-ac1b-4b01-b543-601e6936cf6c admin admin] PATH_INFO: `/v3/projects/a1cc57b4b3c9470f99fd8fa215f03513/users/8e49b1e6d3394ccd839e12ad0683eda9/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:12.245686 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb5e72a-ac1b-4b01-b543-601e6936cf6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1cc57b4b3c9470f99fd8fa215f03513, user_id=8e49b1e6d3394ccd839e12ad0683eda9, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:22:12.248300 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb5e72a-ac1b-4b01-b543-601e6936cf6c admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:22:12.322996 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1484/2940] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 07:22:12 2022] PUT /identity/v3/projects/a1cc57b4b3c9470f99fd8fa215f03513/users/8e49b1e6d3394ccd839e12ad0683eda9/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:22:12.333633 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e4b78-feaf-472b-9d95-89f9ccf18e1f None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:12.333825 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e4b78-feaf-472b-9d95-89f9ccf18e1f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:12.334107 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e4b78-feaf-472b-9d95-89f9ccf18e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:12.357363 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:22:12.360815 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-fa3e4b78-feaf-472b-9d95-89f9ccf18e1f None None] MFA Rules not processed for user `8e49b1e6d3394ccd839e12ad0683eda9`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:22:12.362037 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-fa3e4b78-feaf-472b-9d95-89f9ccf18e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:22:12.426290 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1457/2941] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:22:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:22:12.430877 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb5e72a-ac1b-4b01-b543-601e6936cf6c admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:22:12.457015 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:22:12.492195 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb5e72a-ac1b-4b01-b543-601e6936cf6c admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:22:12.496912 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62077abf-fea0-44bc-96c4-22c973f64413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:22:12.497566 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62077abf-fea0-44bc-96c4-22c973f64413 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:12.497856 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62077abf-fea0-44bc-96c4-22c973f64413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:12.498136 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62077abf-fea0-44bc-96c4-22c973f64413 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:12.498879 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62077abf-fea0-44bc-96c4-22c973f64413 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:22:12.501419 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62077abf-fea0-44bc-96c4-22c973f64413 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:22:12.578860 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1485/2942] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 07:22:12 2022] POST /identity/v3/projects => generated 383 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 07:22:12.582934 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e4b78-feaf-472b-9d95-89f9ccf18e1f None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:22:12.594340 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e4b78-feaf-472b-9d95-89f9ccf18e1f None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:22:12.650580 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534eb5ab-8410-4acb-adaa-80448a0ee940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:22:12.651257 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534eb5ab-8410-4acb-adaa-80448a0ee940 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:12.651548 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534eb5ab-8410-4acb-adaa-80448a0ee940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:12.651842 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534eb5ab-8410-4acb-adaa-80448a0ee940 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:12.652581 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534eb5ab-8410-4acb-adaa-80448a0ee940 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:22:12.654527 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534eb5ab-8410-4acb-adaa-80448a0ee940 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:22:12.748816 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1458/2943] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 07:22:12 2022] POST /identity/v3/users => generated 298 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 07:22:12.753106 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62077abf-fea0-44bc-96c4-22c973f64413 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:22:12.763642 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62077abf-fea0-44bc-96c4-22c973f64413 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:22:12.768613 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd26bfef-3ebe-4a87-9c0c-bb2fdf4d22f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:22:12.769277 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26bfef-3ebe-4a87-9c0c-bb2fdf4d22f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:12.769565 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26bfef-3ebe-4a87-9c0c-bb2fdf4d22f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:12.769849 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26bfef-3ebe-4a87-9c0c-bb2fdf4d22f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:12.770647 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd26bfef-3ebe-4a87-9c0c-bb2fdf4d22f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:22:12.772235 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd26bfef-3ebe-4a87-9c0c-bb2fdf4d22f7 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:22:12.783203 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1486/2944] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:22:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:22:12.787248 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534eb5ab-8410-4acb-adaa-80448a0ee940 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:22:12.798000 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534eb5ab-8410-4acb-adaa-80448a0ee940 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:22:12.802592 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff029e-d27c-441f-8e5e-5545992d7e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:22:12.803469 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff029e-d27c-441f-8e5e-5545992d7e6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:12.803757 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff029e-d27c-441f-8e5e-5545992d7e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:12.804037 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff029e-d27c-441f-8e5e-5545992d7e6e admin admin] PATH_INFO: `/v3/projects/1531b0274472482fa54bd3dc4ef7c561/users/cd3363de39ef451b835642f56e38e8a6/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:12.818507 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff029e-d27c-441f-8e5e-5545992d7e6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1531b0274472482fa54bd3dc4ef7c561, user_id=cd3363de39ef451b835642f56e38e8a6, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:22:12.821463 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff029e-d27c-441f-8e5e-5545992d7e6e admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:22:12.894895 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1459/2945] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 07:22:12 2022] PUT /identity/v3/projects/1531b0274472482fa54bd3dc4ef7c561/users/cd3363de39ef451b835642f56e38e8a6/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:22:12.898915 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd26bfef-3ebe-4a87-9c0c-bb2fdf4d22f7 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:22:12.909074 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd26bfef-3ebe-4a87-9c0c-bb2fdf4d22f7 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:22:12.965554 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b31697-eeec-4eab-a594-545429d13051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:22:12.966149 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b31697-eeec-4eab-a594-545429d13051 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:12.966440 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b31697-eeec-4eab-a594-545429d13051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:12.966724 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b31697-eeec-4eab-a594-545429d13051 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:12.967455 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b31697-eeec-4eab-a594-545429d13051 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:22:12.969132 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b31697-eeec-4eab-a594-545429d13051 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:22:12.978931 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1487/2946] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:22:12 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:22:12.983020 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff029e-d27c-441f-8e5e-5545992d7e6e admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:22:12.994160 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff029e-d27c-441f-8e5e-5545992d7e6e admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:22:12.998736 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9835db-9785-4518-be4c-5525373a8e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:22:13.000592 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9835db-9785-4518-be4c-5525373a8e98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:13.000883 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9835db-9785-4518-be4c-5525373a8e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:13.001168 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9835db-9785-4518-be4c-5525373a8e98 admin admin] PATH_INFO: `/v3/projects/1531b0274472482fa54bd3dc4ef7c561/users/cd3363de39ef451b835642f56e38e8a6/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:13.003155 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9835db-9785-4518-be4c-5525373a8e98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1531b0274472482fa54bd3dc4ef7c561, user_id=cd3363de39ef451b835642f56e38e8a6, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:22:13.005844 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9835db-9785-4518-be4c-5525373a8e98 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:22:13.015864 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1460/2947] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 07:22:12 2022] PUT /identity/v3/projects/1531b0274472482fa54bd3dc4ef7c561/users/cd3363de39ef451b835642f56e38e8a6/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:22:13.026005 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e90f7d-43bd-4495-b844-63fc69c2c0fd None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:13.026301 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e90f7d-43bd-4495-b844-63fc69c2c0fd None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:13.026583 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e90f7d-43bd-4495-b844-63fc69c2c0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:13.052805 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-95e90f7d-43bd-4495-b844-63fc69c2c0fd None None] MFA Rules not processed for user `cd3363de39ef451b835642f56e38e8a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:22:13.054017 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-95e90f7d-43bd-4495-b844-63fc69c2c0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:22:13.122962 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1488/2948] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:22:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 07:22:13.134287 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9835db-9785-4518-be4c-5525373a8e98 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:22:13.135998 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-0e9835db-9785-4518-be4c-5525373a8e98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:22:13.193174 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9835db-9785-4518-be4c-5525373a8e98 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:22:13.197126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1309760-215d-4502-b66d-401f1c903a5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:22:13.197795 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1309760-215d-4502-b66d-401f1c903a5c service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:13.198084 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1309760-215d-4502-b66d-401f1c903a5c service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:13.198364 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1309760-215d-4502-b66d-401f1c903a5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:13.206781 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:22:13.255325 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1309760-215d-4502-b66d-401f1c903a5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:22:13.256797 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1309760-215d-4502-b66d-401f1c903a5c service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:22:13.265758 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1461/2949] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:22:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:22:19.448096 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e90f7d-43bd-4495-b844-63fc69c2c0fd None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:22:19.459041 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e90f7d-43bd-4495-b844-63fc69c2c0fd None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:22:19.462649 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed5a04-cc4a-496d-86f5-b7516edfde78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af3a4a1bc1f4c4e9653d0a790f12606', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:22:19.463295 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed5a04-cc4a-496d-86f5-b7516edfde78 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:19.463583 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed5a04-cc4a-496d-86f5-b7516edfde78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:19.463867 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed5a04-cc4a-496d-86f5-b7516edfde78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:19.472946 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaed5a04-cc4a-496d-86f5-b7516edfde78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:22:19.474377 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaed5a04-cc4a-496d-86f5-b7516edfde78 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:22:19.483992 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1489/2950] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:22:19 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:22:23.269336 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=23 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:22:25.297425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1309760-215d-4502-b66d-401f1c903a5c service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:22:25.359559 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1309760-215d-4502-b66d-401f1c903a5c service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:22:25.364162 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8610f482-e3e9-4348-ac69-6b96de41e7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:22:25.365051 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8610f482-e3e9-4348-ac69-6b96de41e7ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:25.365344 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8610f482-e3e9-4348-ac69-6b96de41e7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:25.365625 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8610f482-e3e9-4348-ac69-6b96de41e7ca admin admin] PATH_INFO: `/v3/users/8e49b1e6d3394ccd839e12ad0683eda9` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:25.366821 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8610f482-e3e9-4348-ac69-6b96de41e7ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e49b1e6d3394ccd839e12ad0683eda9)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:22:25.368767 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8610f482-e3e9-4348-ac69-6b96de41e7ca admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:22:25.597084 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-8610f482-e3e9-4348-ac69-6b96de41e7ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e49b1e6d3394ccd839e12ad0683eda9'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:25.604272 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-8610f482-e3e9-4348-ac69-6b96de41e7ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e49b1e6d3394ccd839e12ad0683eda9'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:25.616642 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-8610f482-e3e9-4348-ac69-6b96de41e7ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e49b1e6d3394ccd839e12ad0683eda9'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:25.617324 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-8610f482-e3e9-4348-ac69-6b96de41e7ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e49b1e6d3394ccd839e12ad0683eda9'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:25.617929 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-8610f482-e3e9-4348-ac69-6b96de41e7ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e49b1e6d3394ccd839e12ad0683eda9'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:25.674845 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1462/2951] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 07:22:25 2022] DELETE /identity/v3/users/8e49b1e6d3394ccd839e12ad0683eda9 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:22:25.679352 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b011a-e76d-40b9-a54a-6b833351bf09 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:25.679796 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b011a-e76d-40b9-a54a-6b833351bf09 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:25.679904 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b011a-e76d-40b9-a54a-6b833351bf09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:25.686507 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:22:25.698035 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-fd0b011a-e76d-40b9-a54a-6b833351bf09 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:22:25.699274 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-fd0b011a-e76d-40b9-a54a-6b833351bf09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:22:25.758177 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1490/2952] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:22:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 07:22:25.768729 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8610f482-e3e9-4348-ac69-6b96de41e7ca admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:22:25.770146 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8610f482-e3e9-4348-ac69-6b96de41e7ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:22:25.827891 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8610f482-e3e9-4348-ac69-6b96de41e7ca admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:22:25.832189 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21aa1d0-c0b4-4a56-bbdb-afa067bec91e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:22:25.832860 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21aa1d0-c0b4-4a56-bbdb-afa067bec91e service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:25.833150 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21aa1d0-c0b4-4a56-bbdb-afa067bec91e service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:25.833432 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21aa1d0-c0b4-4a56-bbdb-afa067bec91e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:25.842398 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21aa1d0-c0b4-4a56-bbdb-afa067bec91e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:22:25.844483 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21aa1d0-c0b4-4a56-bbdb-afa067bec91e service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:22:25.855027 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1463/2953] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:22:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:22:26.254127 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b011a-e76d-40b9-a54a-6b833351bf09 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:22:26.255615 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-fd0b011a-e76d-40b9-a54a-6b833351bf09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:22:26.270149 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b011a-e76d-40b9-a54a-6b833351bf09 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:22:26.274759 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2383810-58ca-4954-8390-050a390d660c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:22:26.276261 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2383810-58ca-4954-8390-050a390d660c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:26.276718 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2383810-58ca-4954-8390-050a390d660c admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:26.276998 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2383810-58ca-4954-8390-050a390d660c admin admin] PATH_INFO: `/v3/projects/a1cc57b4b3c9470f99fd8fa215f03513` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:26.278199 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2383810-58ca-4954-8390-050a390d660c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1cc57b4b3c9470f99fd8fa215f03513)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:22:26.280175 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2383810-58ca-4954-8390-050a390d660c admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:22:26.415277 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d2383810-58ca-4954-8390-050a390d660c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1cc57b4b3c9470f99fd8fa215f03513', 'request_id': 'req-d2383810-58ca-4954-8390-050a390d660c'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:26.421411 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d2383810-58ca-4954-8390-050a390d660c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1cc57b4b3c9470f99fd8fa215f03513', 'request_id': 'req-d2383810-58ca-4954-8390-050a390d660c'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:26.511788 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d2383810-58ca-4954-8390-050a390d660c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1cc57b4b3c9470f99fd8fa215f03513', 'request_id': 'req-d2383810-58ca-4954-8390-050a390d660c'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:26.522866 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d2383810-58ca-4954-8390-050a390d660c admin admin] The token cache is being invalidate because project a1cc57b4b3c9470f99fd8fa215f03513 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 07:22:26.523158 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d2383810-58ca-4954-8390-050a390d660c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:26.523791 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d2383810-58ca-4954-8390-050a390d660c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:26.525354 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1491/2954] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 07:22:26 2022] DELETE /identity/v3/projects/a1cc57b4b3c9470f99fd8fa215f03513 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:22:26.529551 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21aa1d0-c0b4-4a56-bbdb-afa067bec91e service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:22:26.539425 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21aa1d0-c0b4-4a56-bbdb-afa067bec91e service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:22:26.558240 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:22:26.595057 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4529fe1-178f-4822-99d4-fd0ad3ba2f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:22:26.596057 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4529fe1-178f-4822-99d4-fd0ad3ba2f68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:26.596353 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4529fe1-178f-4822-99d4-fd0ad3ba2f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:26.596856 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4529fe1-178f-4822-99d4-fd0ad3ba2f68 admin admin] PATH_INFO: `/v3/users/cd3363de39ef451b835642f56e38e8a6` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:26.598446 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4529fe1-178f-4822-99d4-fd0ad3ba2f68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd3363de39ef451b835642f56e38e8a6)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:22:26.600380 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4529fe1-178f-4822-99d4-fd0ad3ba2f68 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:22:26.774539 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f4529fe1-178f-4822-99d4-fd0ad3ba2f68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd3363de39ef451b835642f56e38e8a6'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:26.781404 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f4529fe1-178f-4822-99d4-fd0ad3ba2f68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd3363de39ef451b835642f56e38e8a6'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:26.793595 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f4529fe1-178f-4822-99d4-fd0ad3ba2f68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd3363de39ef451b835642f56e38e8a6'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:26.794264 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f4529fe1-178f-4822-99d4-fd0ad3ba2f68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd3363de39ef451b835642f56e38e8a6'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:26.794868 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f4529fe1-178f-4822-99d4-fd0ad3ba2f68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd3363de39ef451b835642f56e38e8a6'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:26.882962 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1464/2955] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 07:22:26 2022] DELETE /identity/v3/users/cd3363de39ef451b835642f56e38e8a6 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:22:27.299890 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2383810-58ca-4954-8390-050a390d660c admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:22:27.301360 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d2383810-58ca-4954-8390-050a390d660c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:22:27.316604 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2383810-58ca-4954-8390-050a390d660c admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:22:27.374408 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dce65bd-2826-4512-980f-92c970a76faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:22:27.375269 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce65bd-2826-4512-980f-92c970a76faf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:27.375554 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce65bd-2826-4512-980f-92c970a76faf admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:27.375831 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce65bd-2826-4512-980f-92c970a76faf admin admin] PATH_INFO: `/v3/projects/1531b0274472482fa54bd3dc4ef7c561` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:27.377593 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dce65bd-2826-4512-980f-92c970a76faf admin admin] RBAC: Authorizing `identity:delete_project(project_id=1531b0274472482fa54bd3dc4ef7c561)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:22:27.379492 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dce65bd-2826-4512-980f-92c970a76faf admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:22:27.482322 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:22:27.495249 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6dce65bd-2826-4512-980f-92c970a76faf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1531b0274472482fa54bd3dc4ef7c561', 'request_id': 'req-6dce65bd-2826-4512-980f-92c970a76faf'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:27.501675 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6dce65bd-2826-4512-980f-92c970a76faf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1531b0274472482fa54bd3dc4ef7c561', 'request_id': 'req-6dce65bd-2826-4512-980f-92c970a76faf'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:27.574724 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6dce65bd-2826-4512-980f-92c970a76faf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1531b0274472482fa54bd3dc4ef7c561', 'request_id': 'req-6dce65bd-2826-4512-980f-92c970a76faf'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:27.585633 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6dce65bd-2826-4512-980f-92c970a76faf admin admin] The token cache is being invalidate because project 1531b0274472482fa54bd3dc4ef7c561 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 07:22:27.594034 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6dce65bd-2826-4512-980f-92c970a76faf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:27.594034 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6dce65bd-2826-4512-980f-92c970a76faf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:22:27.594034 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1492/2956] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 07:22:27 2022] DELETE /identity/v3/projects/1531b0274472482fa54bd3dc4ef7c561 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:22:27.594459 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff7382-48db-46a9-b234-4cf44fa36857 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:27.595427 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff7382-48db-46a9-b234-4cf44fa36857 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:27.595588 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff7382-48db-46a9-b234-4cf44fa36857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:27.615172 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-83ff7382-48db-46a9-b234-4cf44fa36857 None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:22:27.616404 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-83ff7382-48db-46a9-b234-4cf44fa36857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:22:27.676751 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1465/2957] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:22:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:22:27.687279 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea8135-cd17-4dc6-8de6-96118f3dd66d None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:27.687279 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea8135-cd17-4dc6-8de6-96118f3dd66d None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:27.687681 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea8135-cd17-4dc6-8de6-96118f3dd66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:27.706598 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-25ea8135-cd17-4dc6-8de6-96118f3dd66d None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:22:27.708265 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-25ea8135-cd17-4dc6-8de6-96118f3dd66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:22:27.716785 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1493/2958] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:22:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:22:27.721713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff7382-48db-46a9-b234-4cf44fa36857 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:22:27.731191 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff7382-48db-46a9-b234-4cf44fa36857 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:22:27.736277 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62787b0-2f86-444e-b3d7-135a05493829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:22:27.736959 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62787b0-2f86-444e-b3d7-135a05493829 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:27.737240 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62787b0-2f86-444e-b3d7-135a05493829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:27.737510 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62787b0-2f86-444e-b3d7-135a05493829 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:27.738230 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62787b0-2f86-444e-b3d7-135a05493829 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 07:22:27.738532 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62787b0-2f86-444e-b3d7-135a05493829 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:22:27.740101 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62787b0-2f86-444e-b3d7-135a05493829 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:22:27.751635 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1466/2959] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 18 07:22:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 07:22:27.756234 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ea8135-cd17-4dc6-8de6-96118f3dd66d None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:22:27.765448 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ea8135-cd17-4dc6-8de6-96118f3dd66d None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:22:27.770351 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e430e-19ad-4669-bb43-c15a86d596aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:22:27.771000 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e430e-19ad-4669-bb43-c15a86d596aa admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:27.771287 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e430e-19ad-4669-bb43-c15a86d596aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:27.771568 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e430e-19ad-4669-bb43-c15a86d596aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:27.772263 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799e430e-19ad-4669-bb43-c15a86d596aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:22:27.774191 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799e430e-19ad-4669-bb43-c15a86d596aa admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:22:27.876256 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1494/2960] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 07:22:27 2022] POST /identity/v3/projects => generated 367 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 07:22:27.880618 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62787b0-2f86-444e-b3d7-135a05493829 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:22:27.892600 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62787b0-2f86-444e-b3d7-135a05493829 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:22:27.918897 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:22:27.948658 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b021f-d086-4566-914a-2f36b27b662f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:22:27.949325 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b021f-d086-4566-914a-2f36b27b662f admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:27.949617 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b021f-d086-4566-914a-2f36b27b662f admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:27.949927 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b021f-d086-4566-914a-2f36b27b662f admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:27.950640 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850b021f-d086-4566-914a-2f36b27b662f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:22:27.952588 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850b021f-d086-4566-914a-2f36b27b662f admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:22:28.037905 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1467/2961] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 07:22:27 2022] POST /identity/v3/users => generated 284 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 07:22:28.041987 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e430e-19ad-4669-bb43-c15a86d596aa admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:22:28.052758 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e430e-19ad-4669-bb43-c15a86d596aa admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:22:28.057500 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635278af-216e-4d52-a8fb-45d7c6865be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:22:28.059026 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635278af-216e-4d52-a8fb-45d7c6865be5 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:28.059026 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635278af-216e-4d52-a8fb-45d7c6865be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:28.059026 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635278af-216e-4d52-a8fb-45d7c6865be5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:28.059486 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635278af-216e-4d52-a8fb-45d7c6865be5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:22:28.061131 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635278af-216e-4d52-a8fb-45d7c6865be5 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:22:28.072052 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1495/2962] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:22:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:22:28.075956 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b021f-d086-4566-914a-2f36b27b662f admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:22:28.086412 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b021f-d086-4566-914a-2f36b27b662f admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:22:28.091355 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9a5e1-1e53-4907-a570-963d79f90afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:22:28.092008 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9a5e1-1e53-4907-a570-963d79f90afe admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:28.092296 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9a5e1-1e53-4907-a570-963d79f90afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:28.092618 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9a5e1-1e53-4907-a570-963d79f90afe admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:28.093454 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f9a5e1-1e53-4907-a570-963d79f90afe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:22:28.096090 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f9a5e1-1e53-4907-a570-963d79f90afe admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:22:28.106504 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1468/2963] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:22:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:22:28.112036 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635278af-216e-4d52-a8fb-45d7c6865be5 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:22:28.120444 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635278af-216e-4d52-a8fb-45d7c6865be5 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:22:28.124979 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165798a-fd10-439e-a71a-2c3ec28a7ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:22:28.125844 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165798a-fd10-439e-a71a-2c3ec28a7ad0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:28.126171 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165798a-fd10-439e-a71a-2c3ec28a7ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:28.126550 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165798a-fd10-439e-a71a-2c3ec28a7ad0 admin admin] PATH_INFO: `/v3/projects/1acd582259d447829d53a5719dd31266/users/ece0f77dbbf1474b8f2205c2cbc7b881/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:28.140919 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8165798a-fd10-439e-a71a-2c3ec28a7ad0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1acd582259d447829d53a5719dd31266, user_id=ece0f77dbbf1474b8f2205c2cbc7b881, role_id=c3ceac8e8c2f47f2a1cfdaf3a24bfe20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:22:28.144067 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8165798a-fd10-439e-a71a-2c3ec28a7ad0 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:22:28.202718 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1496/2964] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 07:22:28 2022] PUT /identity/v3/projects/1acd582259d447829d53a5719dd31266/users/ece0f77dbbf1474b8f2205c2cbc7b881/roles/c3ceac8e8c2f47f2a1cfdaf3a24bfe20 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:22:28.208611 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5e876-5be5-41a7-bbff-f84c361a8ece None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:28.208928 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5e876-5be5-41a7-bbff-f84c361a8ece None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:28.209210 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5e876-5be5-41a7-bbff-f84c361a8ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:28.227032 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-eea5e876-5be5-41a7-bbff-f84c361a8ece None None] MFA Rules not processed for user `8783051891e642af84b75a46b4d6b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:22:28.228268 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-eea5e876-5be5-41a7-bbff-f84c361a8ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:22:28.287067 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1469/2965] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:22:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:22:28.296816 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165798a-fd10-439e-a71a-2c3ec28a7ad0 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:22:28.298509 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-8165798a-fd10-439e-a71a-2c3ec28a7ad0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:22:28.358230 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165798a-fd10-439e-a71a-2c3ec28a7ad0 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:22:28.362242 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef65e9-74bf-4090-bbee-8b06246f6c73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:22:28.362883 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef65e9-74bf-4090-bbee-8b06246f6c73 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:28.363171 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef65e9-74bf-4090-bbee-8b06246f6c73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:28.363451 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef65e9-74bf-4090-bbee-8b06246f6c73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:28.372554 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef65e9-74bf-4090-bbee-8b06246f6c73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:22:28.373979 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef65e9-74bf-4090-bbee-8b06246f6c73 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:22:28.383865 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1497/2966] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:22:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:22:33.207161 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c273e65-5d01-4260-8e7e-33b209a08dfa None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:33.207689 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c273e65-5d01-4260-8e7e-33b209a08dfa None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:33.207689 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c273e65-5d01-4260-8e7e-33b209a08dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:33.237584 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-4c273e65-5d01-4260-8e7e-33b209a08dfa None None] MFA Rules not processed for user `ece0f77dbbf1474b8f2205c2cbc7b881`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:22:33.239649 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4c273e65-5d01-4260-8e7e-33b209a08dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:22:33.311049 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1470/2967] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:22:33 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:22:33.316627 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef65e9-74bf-4090-bbee-8b06246f6c73 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:22:33.327413 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef65e9-74bf-4090-bbee-8b06246f6c73 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:22:33.332623 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c95ae7-f049-4f6c-b32e-6b9ef4352540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:22:33.333309 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c95ae7-f049-4f6c-b32e-6b9ef4352540 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:33.333758 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c95ae7-f049-4f6c-b32e-6b9ef4352540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:33.334039 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c95ae7-f049-4f6c-b32e-6b9ef4352540 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:33.334753 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c95ae7-f049-4f6c-b32e-6b9ef4352540 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:22:33.336683 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c95ae7-f049-4f6c-b32e-6b9ef4352540 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:22:33.428093 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1498/2968] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 18 07:22:33 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 07:22:33.432142 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c273e65-5d01-4260-8e7e-33b209a08dfa None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:22:33.443959 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c273e65-5d01-4260-8e7e-33b209a08dfa None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:22:33.478247 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:22:33.501687 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b435af-f333-4c3e-91cd-b1e2235e276e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:22:33.502267 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b435af-f333-4c3e-91cd-b1e2235e276e admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:33.502553 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b435af-f333-4c3e-91cd-b1e2235e276e admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:33.502837 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b435af-f333-4c3e-91cd-b1e2235e276e admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:33.503523 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b435af-f333-4c3e-91cd-b1e2235e276e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:22:33.505560 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b435af-f333-4c3e-91cd-b1e2235e276e admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:22:33.604179 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1471/2969] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 18 07:22:33 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 07:22:33.608117 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c95ae7-f049-4f6c-b32e-6b9ef4352540 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:22:33.623366 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c95ae7-f049-4f6c-b32e-6b9ef4352540 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:22:33.631633 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65151d5-7401-49ac-b651-7b266c3f29e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:22:33.632619 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65151d5-7401-49ac-b651-7b266c3f29e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:33.633016 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65151d5-7401-49ac-b651-7b266c3f29e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:33.633402 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65151d5-7401-49ac-b651-7b266c3f29e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:33.634301 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65151d5-7401-49ac-b651-7b266c3f29e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:22:33.636663 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65151d5-7401-49ac-b651-7b266c3f29e9 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:22:33.648936 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1499/2970] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:22:33.652883 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b435af-f333-4c3e-91cd-b1e2235e276e admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:22:33.664020 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b435af-f333-4c3e-91cd-b1e2235e276e admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:22:33.668676 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b33be4-dae9-4019-a9e7-4cb536176693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:22:33.669559 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b33be4-dae9-4019-a9e7-4cb536176693 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:33.669848 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b33be4-dae9-4019-a9e7-4cb536176693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:33.670126 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b33be4-dae9-4019-a9e7-4cb536176693 admin admin] PATH_INFO: `/v3/projects/55849460624d495fa2c415ade0e707a2/users/fdd45c60e092432a8f97b521246aa229/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:33.686432 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b33be4-dae9-4019-a9e7-4cb536176693 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55849460624d495fa2c415ade0e707a2, user_id=fdd45c60e092432a8f97b521246aa229, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:22:33.689728 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b33be4-dae9-4019-a9e7-4cb536176693 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:22:33.759332 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1472/2971] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 07:22:33 2022] PUT /identity/v3/projects/55849460624d495fa2c415ade0e707a2/users/fdd45c60e092432a8f97b521246aa229/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 07:22:33.763500 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65151d5-7401-49ac-b651-7b266c3f29e9 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:22:33.774466 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65151d5-7401-49ac-b651-7b266c3f29e9 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:22:33.799092 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:22:33.843616 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a3ca9-6f5c-4f0d-8157-e68449a547a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:22:33.844294 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a3ca9-6f5c-4f0d-8157-e68449a547a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:33.844622 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a3ca9-6f5c-4f0d-8157-e68449a547a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:33.845519 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a3ca9-6f5c-4f0d-8157-e68449a547a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:33.846249 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a3ca9-6f5c-4f0d-8157-e68449a547a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:22:33.847914 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a3ca9-6f5c-4f0d-8157-e68449a547a8 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:22:33.859386 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1500/2972] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 18 07:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 07:22:33.863535 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b33be4-dae9-4019-a9e7-4cb536176693 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:22:33.874190 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b33be4-dae9-4019-a9e7-4cb536176693 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:22:33.879166 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd8c1e-b7ce-484c-9fb1-e9c041ca8a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:22:33.880084 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd8c1e-b7ce-484c-9fb1-e9c041ca8a47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:33.880370 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd8c1e-b7ce-484c-9fb1-e9c041ca8a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:33.880682 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd8c1e-b7ce-484c-9fb1-e9c041ca8a47 admin admin] PATH_INFO: `/v3/projects/55849460624d495fa2c415ade0e707a2/users/fdd45c60e092432a8f97b521246aa229/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:33.882749 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fd8c1e-b7ce-484c-9fb1-e9c041ca8a47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55849460624d495fa2c415ade0e707a2, user_id=fdd45c60e092432a8f97b521246aa229, role_id=13e6e9f5da2c48f8ab6915c9f3dd3fdc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:22:33.885457 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fd8c1e-b7ce-484c-9fb1-e9c041ca8a47 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:22:33.901520 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1473/2973] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 18 07:22:33 2022] PUT /identity/v3/projects/55849460624d495fa2c415ade0e707a2/users/fdd45c60e092432a8f97b521246aa229/roles/13e6e9f5da2c48f8ab6915c9f3dd3fdc => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:22:39.348092 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2eeced-aab1-4c26-971d-ac6297a4c4ff None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:39.348742 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2eeced-aab1-4c26-971d-ac6297a4c4ff None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:39.348742 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2eeced-aab1-4c26-971d-ac6297a4c4ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:39.376262 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-1f2eeced-aab1-4c26-971d-ac6297a4c4ff None None] MFA Rules not processed for user `fdd45c60e092432a8f97b521246aa229`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:22:39.377548 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-1f2eeced-aab1-4c26-971d-ac6297a4c4ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:22:39.448703 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1501/2974] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:22:39 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 07:22:39.466258 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd8c1e-b7ce-484c-9fb1-e9c041ca8a47 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:22:39.471004 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-51fd8c1e-b7ce-484c-9fb1-e9c041ca8a47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:22:39.550100 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd8c1e-b7ce-484c-9fb1-e9c041ca8a47 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:22:39.556021 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394321c1-9be6-4363-974b-c3b9d778aec6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:22:39.556709 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394321c1-9be6-4363-974b-c3b9d778aec6 service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:39.557007 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394321c1-9be6-4363-974b-c3b9d778aec6 service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:39.557397 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394321c1-9be6-4363-974b-c3b9d778aec6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:39.614320 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394321c1-9be6-4363-974b-c3b9d778aec6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:22:39.615956 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394321c1-9be6-4363-974b-c3b9d778aec6 service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:22:39.626555 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1474/2975] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:22:39 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:22:42.341567 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e992b-6d3b-4ba9-9f54-43b4ab04ae6e None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:22:42.342051 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e992b-6d3b-4ba9-9f54-43b4ab04ae6e None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:22:42.342158 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e992b-6d3b-4ba9-9f54-43b4ab04ae6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:22:42.364593 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-190e992b-6d3b-4ba9-9f54-43b4ab04ae6e None None] MFA Rules not processed for user `ece0f77dbbf1474b8f2205c2cbc7b881`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:22:42.366589 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-190e992b-6d3b-4ba9-9f54-43b4ab04ae6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:22:42.374555 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1502/2976] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 18 07:22:42 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:22:49.621554 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=83,keystone:INSERT=5 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:22:52.357299 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=55 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:23:08.577059 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394321c1-9be6-4363-974b-c3b9d778aec6 service nova] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:23:08.631893 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394321c1-9be6-4363-974b-c3b9d778aec6 service nova] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:23:08.635457 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd46bc-bbbe-46ed-87ca-c9cc96610d15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:23:08.636094 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd46bc-bbbe-46ed-87ca-c9cc96610d15 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:23:08.636381 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd46bc-bbbe-46ed-87ca-c9cc96610d15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:23:08.636703 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd46bc-bbbe-46ed-87ca-c9cc96610d15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:23:08.638479 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-53cd46bc-bbbe-46ed-87ca-c9cc96610d15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:23:08.647988 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cd46bc-bbbe-46ed-87ca-c9cc96610d15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:23:08.649439 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cd46bc-bbbe-46ed-87ca-c9cc96610d15 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:23:08.657543 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1475/2977] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:23:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:23:09.381265 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190e992b-6d3b-4ba9-9f54-43b4ab04ae6e None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:23:09.382684 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-190e992b-6d3b-4ba9-9f54-43b4ab04ae6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:23:09.453726 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190e992b-6d3b-4ba9-9f54-43b4ab04ae6e None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:23:09.458622 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f1b54-2f35-47c6-b0f8-addf9505a244 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3560534427544cdabfd0ba702a0a165', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:23:09.459428 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f1b54-2f35-47c6-b0f8-addf9505a244 service placement] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:23:09.459707 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f1b54-2f35-47c6-b0f8-addf9505a244 service placement] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:23:09.459980 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f1b54-2f35-47c6-b0f8-addf9505a244 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:23:09.461802 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-223f1b54-2f35-47c6-b0f8-addf9505a244 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:23:09.472748 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223f1b54-2f35-47c6-b0f8-addf9505a244 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:23:09.474918 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223f1b54-2f35-47c6-b0f8-addf9505a244 service placement] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:23:09.484417 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1503/2978] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:23:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 07:23:18.653848 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=26 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:23:19.480450 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=32 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:23:30.585552 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd46bc-bbbe-46ed-87ca-c9cc96610d15 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:23:30.594021 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd46bc-bbbe-46ed-87ca-c9cc96610d15 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:23:30.597612 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4e5264-7aed-4f81-896b-0d8b6743033a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:23:30.598248 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e5264-7aed-4f81-896b-0d8b6743033a service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:23:30.598538 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e5264-7aed-4f81-896b-0d8b6743033a service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:23:30.598820 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e5264-7aed-4f81-896b-0d8b6743033a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:23:30.608225 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4e5264-7aed-4f81-896b-0d8b6743033a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:23:30.609690 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4e5264-7aed-4f81-896b-0d8b6743033a service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:23:30.619681 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1476/2979] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:23:30 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:23:40.614877 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=6 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:23:52.901442 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f1b54-2f35-47c6-b0f8-addf9505a244 service placement] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:23:52.910575 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f1b54-2f35-47c6-b0f8-addf9505a244 service placement] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:23:52.914131 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5a6d5-6cd4-44ef-be13-771d9901d6fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:23:52.914767 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5a6d5-6cd4-44ef-be13-771d9901d6fa service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:23:52.915069 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5a6d5-6cd4-44ef-be13-771d9901d6fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:23:52.915416 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5a6d5-6cd4-44ef-be13-771d9901d6fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:23:52.917262 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-8af5a6d5-6cd4-44ef-be13-771d9901d6fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:23:52.926614 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af5a6d5-6cd4-44ef-be13-771d9901d6fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:23:52.928037 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af5a6d5-6cd4-44ef-be13-771d9901d6fa service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:23:52.936140 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1504/2980] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:23:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:24:02.932555 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=6 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:24:53.895282 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3bd7d-6f59-44e4-b787-87405b6303bb None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:24:53.895860 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3bd7d-6f59-44e4-b787-87405b6303bb None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:24:53.895860 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3bd7d-6f59-44e4-b787-87405b6303bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:24:53.914320 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-48c3bd7d-6f59-44e4-b787-87405b6303bb None None] MFA Rules not processed for user `9824a96e2bcd4a16bfc6fe6bf4ea1ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:24:53.916181 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-48c3bd7d-6f59-44e4-b787-87405b6303bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:24:53.922925 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1477/2981] 10.222.0.17 () {58 vars in 1015 bytes} [Mon Jul 18 07:24:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:24:53.936217 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5a6d5-6cd4-44ef-be13-771d9901d6fa service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:24:53.945053 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5a6d5-6cd4-44ef-be13-771d9901d6fa service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:24:53.948717 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757cd99-211b-4cd7-b2c5-3ccce0224bba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:24:53.949351 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757cd99-211b-4cd7-b2c5-3ccce0224bba service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:24:53.949635 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757cd99-211b-4cd7-b2c5-3ccce0224bba service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:24:53.949910 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757cd99-211b-4cd7-b2c5-3ccce0224bba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:24:53.956809 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e757cd99-211b-4cd7-b2c5-3ccce0224bba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:24:53.958215 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e757cd99-211b-4cd7-b2c5-3ccce0224bba service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:24:53.966358 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1505/2982] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:24:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 31 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:25:03.909049 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:25:03.962969 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=6 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:25:46.737305 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829275a-9072-4a9a-a225-9714eaacc5b0 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:25:46.737898 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829275a-9072-4a9a-a225-9714eaacc5b0 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:25:46.737898 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829275a-9072-4a9a-a225-9714eaacc5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:25:46.776843 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-3829275a-9072-4a9a-a225-9714eaacc5b0 None None] MFA Rules not processed for user `f92d948810ea4241810473506b654f7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:25:46.778100 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-3829275a-9072-4a9a-a225-9714eaacc5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:25:46.787299 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1478/2983] 10.222.0.40 () {58 vars in 1059 bytes} [Mon Jul 18 07:25:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:25:46.793334 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e60d33-fd34-464a-ad0f-6639389d2dfe None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:25:46.793865 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e60d33-fd34-464a-ad0f-6639389d2dfe None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:25:46.793963 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e60d33-fd34-464a-ad0f-6639389d2dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:25:46.812239 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-12e60d33-fd34-464a-ad0f-6639389d2dfe None None] MFA Rules not processed for user `f92d948810ea4241810473506b654f7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:25:46.813419 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-12e60d33-fd34-464a-ad0f-6639389d2dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:25:46.823471 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1506/2984] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Jul 18 07:25:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 07:25:46.833698 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3829275a-9072-4a9a-a225-9714eaacc5b0 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:25:46.842972 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3829275a-9072-4a9a-a225-9714eaacc5b0 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:25:46.846561 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c3ebc-aa84-42df-8d14-979181d41d31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:25:46.847216 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c3ebc-aa84-42df-8d14-979181d41d31 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:25:46.847487 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c3ebc-aa84-42df-8d14-979181d41d31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:25:46.847766 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c3ebc-aa84-42df-8d14-979181d41d31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:25:46.857872 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0c3ebc-aa84-42df-8d14-979181d41d31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:25:46.859318 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0c3ebc-aa84-42df-8d14-979181d41d31 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:25:46.870613 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1479/2985] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:25:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:25:48.040011 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ded6d2-7983-4e8e-9738-127d4a2dc410 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:25:48.040608 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ded6d2-7983-4e8e-9738-127d4a2dc410 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:25:48.040727 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ded6d2-7983-4e8e-9738-127d4a2dc410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:25:48.058349 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-a5ded6d2-7983-4e8e-9738-127d4a2dc410 None None] MFA Rules not processed for user `f92d948810ea4241810473506b654f7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:25:48.059547 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-a5ded6d2-7983-4e8e-9738-127d4a2dc410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:25:48.069154 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1507/2986] 10.222.0.40 () {58 vars in 1059 bytes} [Mon Jul 18 07:25:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:25:48.075112 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d329ac-5fd1-4693-bed9-06815e2f422a None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:25:48.075595 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d329ac-5fd1-4693-bed9-06815e2f422a None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:25:48.075696 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d329ac-5fd1-4693-bed9-06815e2f422a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:25:48.094160 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-83d329ac-5fd1-4693-bed9-06815e2f422a None None] MFA Rules not processed for user `f92d948810ea4241810473506b654f7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:25:48.095318 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-83d329ac-5fd1-4693-bed9-06815e2f422a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:25:48.104568 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1480/2987] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Jul 18 07:25:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:25:48.114954 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ded6d2-7983-4e8e-9738-127d4a2dc410 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:25:48.124501 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ded6d2-7983-4e8e-9738-127d4a2dc410 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:25:48.128030 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325cddd6-620d-4dd6-8893-e160e31c739e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:25:48.128705 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325cddd6-620d-4dd6-8893-e160e31c739e service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:25:48.128989 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325cddd6-620d-4dd6-8893-e160e31c739e service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:25:48.129264 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325cddd6-620d-4dd6-8893-e160e31c739e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:25:48.139850 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325cddd6-620d-4dd6-8893-e160e31c739e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:25:48.141317 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325cddd6-620d-4dd6-8893-e160e31c739e service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:25:48.152703 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1508/2988] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:25:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:25:58.089350 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=18 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:25:58.147254 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=12 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:26:00.302193 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b7719-bd4d-4963-8ac7-421d28a75379 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:26:00.302731 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b7719-bd4d-4963-8ac7-421d28a75379 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:26:00.302856 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b7719-bd4d-4963-8ac7-421d28a75379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:26:00.348909 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-3d0b7719-bd4d-4963-8ac7-421d28a75379 None None] MFA Rules not processed for user `9824a96e2bcd4a16bfc6fe6bf4ea1ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:26:00.350117 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-3d0b7719-bd4d-4963-8ac7-421d28a75379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:26:00.365862 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1481/2989] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Jul 18 07:26:00 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:26:00.377793 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325cddd6-620d-4dd6-8893-e160e31c739e service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:26:00.387301 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325cddd6-620d-4dd6-8893-e160e31c739e service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:26:00.391054 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3193daa2-28f0-41fe-9950-feaa498878cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:26:00.391690 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193daa2-28f0-41fe-9950-feaa498878cf service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:26:00.391975 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193daa2-28f0-41fe-9950-feaa498878cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:26:00.392253 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193daa2-28f0-41fe-9950-feaa498878cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:26:00.400748 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3193daa2-28f0-41fe-9950-feaa498878cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:26:00.402271 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3193daa2-28f0-41fe-9950-feaa498878cf service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:26:00.409999 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1509/2990] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:26:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:26:00.722797 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef26ae-8a55-41da-b6bb-9f3b484b74a1 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:26:00.723157 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef26ae-8a55-41da-b6bb-9f3b484b74a1 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:26:00.723365 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef26ae-8a55-41da-b6bb-9f3b484b74a1 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:26:00.724782 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1482/2991] 10.222.0.40 () {54 vars in 921 bytes} [Mon Jul 18 07:26:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 07:26:00.730905 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68438f06-eda8-44e2-a17b-f896fe5c0ac5 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:26:00.731199 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68438f06-eda8-44e2-a17b-f896fe5c0ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:26:00.731484 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68438f06-eda8-44e2-a17b-f896fe5c0ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:26:00.750016 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-68438f06-eda8-44e2-a17b-f896fe5c0ac5 None None] MFA Rules not processed for user `9824a96e2bcd4a16bfc6fe6bf4ea1ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:26:00.751894 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-68438f06-eda8-44e2-a17b-f896fe5c0ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:26:00.758100 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1510/2992] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Jul 18 07:26:00 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:26:00.770559 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef26ae-8a55-41da-b6bb-9f3b484b74a1 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:26:00.772961 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-0cef26ae-8a55-41da-b6bb-9f3b484b74a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:26:00.787060 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef26ae-8a55-41da-b6bb-9f3b484b74a1 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:26:00.790590 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433e1f0-dc45-41be-af76-f121efd560ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:26:00.801174 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433e1f0-dc45-41be-af76-f121efd560ca service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:26:00.801472 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433e1f0-dc45-41be-af76-f121efd560ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:26:00.802115 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433e1f0-dc45-41be-af76-f121efd560ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:26:00.814410 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b433e1f0-dc45-41be-af76-f121efd560ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:26:00.814410 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b433e1f0-dc45-41be-af76-f121efd560ca service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:26:00.820744 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1483/2993] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:26:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:26:00.904249 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad5ed4-1182-445c-a41a-5a83a019259c None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:26:00.904569 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad5ed4-1182-445c-a41a-5a83a019259c None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:26:00.904851 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad5ed4-1182-445c-a41a-5a83a019259c None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:26:00.906218 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1511/2994] 10.222.0.40 () {54 vars in 921 bytes} [Mon Jul 18 07:26:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 07:26:00.911691 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44d5ea-09fa-4385-9c44-de62201f45f0 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:26:00.911987 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44d5ea-09fa-4385-9c44-de62201f45f0 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:26:00.912270 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44d5ea-09fa-4385-9c44-de62201f45f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:26:00.931317 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-9a44d5ea-09fa-4385-9c44-de62201f45f0 None None] MFA Rules not processed for user `f92d948810ea4241810473506b654f7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:26:00.932584 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-9a44d5ea-09fa-4385-9c44-de62201f45f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:26:00.942003 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1484/2995] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Jul 18 07:26:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:26:00.952491 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ad5ed4-1182-445c-a41a-5a83a019259c None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:26:00.963659 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ad5ed4-1182-445c-a41a-5a83a019259c None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:26:00.967198 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c6d21-e37d-4cf0-af6e-09f2123967ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:26:00.967821 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c6d21-e37d-4cf0-af6e-09f2123967ce service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:26:00.968104 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c6d21-e37d-4cf0-af6e-09f2123967ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:26:00.968378 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c6d21-e37d-4cf0-af6e-09f2123967ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:26:00.978347 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05c6d21-e37d-4cf0-af6e-09f2123967ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:26:00.980453 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05c6d21-e37d-4cf0-af6e-09f2123967ce service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:26:00.991038 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1512/2996] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:26:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 07:26:01.051062 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a44d5ea-09fa-4385-9c44-de62201f45f0 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:26:01.052476 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-9a44d5ea-09fa-4385-9c44-de62201f45f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:26:01.075754 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a44d5ea-09fa-4385-9c44-de62201f45f0 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:26:01.080728 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d240de-baa3-4a34-8e9a-6d19dcb43424 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3560534427544cdabfd0ba702a0a165', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:26:01.081365 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d240de-baa3-4a34-8e9a-6d19dcb43424 service placement] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:26:01.081654 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d240de-baa3-4a34-8e9a-6d19dcb43424 service placement] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:26:01.081950 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d240de-baa3-4a34-8e9a-6d19dcb43424 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:26:01.083696 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-49d240de-baa3-4a34-8e9a-6d19dcb43424 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:26:01.095127 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d240de-baa3-4a34-8e9a-6d19dcb43424 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:26:01.096716 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d240de-baa3-4a34-8e9a-6d19dcb43424 service placement] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:26:01.106946 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1485/2997] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:26:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:26:03.430016 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a69f80-7633-4a75-a2a0-08d98fce99f7 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:26:03.435489 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a69f80-7633-4a75-a2a0-08d98fce99f7 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:26:03.435489 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a69f80-7633-4a75-a2a0-08d98fce99f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:26:03.448869 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-86a69f80-7633-4a75-a2a0-08d98fce99f7 None None] MFA Rules not processed for user `f92d948810ea4241810473506b654f7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:26:03.450079 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-86a69f80-7633-4a75-a2a0-08d98fce99f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:26:03.459116 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1513/2998] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Jul 18 07:26:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:26:03.471776 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d240de-baa3-4a34-8e9a-6d19dcb43424 service placement] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:26:03.484578 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d240de-baa3-4a34-8e9a-6d19dcb43424 service placement] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:26:03.490015 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd31da8-1225-4b95-9f6a-f4a1227febb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:26:03.496226 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd31da8-1225-4b95-9f6a-f4a1227febb4 service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:26:03.496226 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd31da8-1225-4b95-9f6a-f4a1227febb4 service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:26:03.496226 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd31da8-1225-4b95-9f6a-f4a1227febb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:26:03.500297 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd31da8-1225-4b95-9f6a-f4a1227febb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:26:03.501914 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd31da8-1225-4b95-9f6a-f4a1227febb4 service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:26:03.512748 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1486/2999] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:26:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:26:13.443839 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=18 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:26:13.507917 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=42 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:27:05.192368 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c18e4-6773-4a8b-b632-938564f11b9a None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:27:05.192830 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c18e4-6773-4a8b-b632-938564f11b9a None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:27:05.192955 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c18e4-6773-4a8b-b632-938564f11b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:27:05.210491 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-4a0c18e4-6773-4a8b-b632-938564f11b9a None None] MFA Rules not processed for user `a3560534427544cdabfd0ba702a0a165`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:27:05.211755 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4a0c18e4-6773-4a8b-b632-938564f11b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:27:05.220281 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1514/3000] 10.222.0.40 () {58 vars in 1058 bytes} [Mon Jul 18 07:27:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:27:05.225957 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd31da8-1225-4b95-9f6a-f4a1227febb4 service nova] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:27:05.237386 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd31da8-1225-4b95-9f6a-f4a1227febb4 service nova] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:27:05.242351 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d452ee4-446d-4383-9f38-9db25f59c249 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3560534427544cdabfd0ba702a0a165', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:27:05.243679 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d452ee4-446d-4383-9f38-9db25f59c249 service placement] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:27:05.243964 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d452ee4-446d-4383-9f38-9db25f59c249 service placement] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:27:05.244240 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d452ee4-446d-4383-9f38-9db25f59c249 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:27:05.246053 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-5d452ee4-446d-4383-9f38-9db25f59c249 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:27:05.257111 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d452ee4-446d-4383-9f38-9db25f59c249 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:27:05.258724 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d452ee4-446d-4383-9f38-9db25f59c249 service placement] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:27:05.268357 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1487/3001] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:27:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:27:15.205493 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:27:15.264288 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=6 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:27:47.121578 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c18e4-6773-4a8b-b632-938564f11b9a None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:27:47.130789 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c18e4-6773-4a8b-b632-938564f11b9a None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:27:47.134326 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880fafe6-0b48-4dd2-b80b-8cbf5c314d2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:27:47.135669 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fafe6-0b48-4dd2-b80b-8cbf5c314d2c service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:27:47.135961 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fafe6-0b48-4dd2-b80b-8cbf5c314d2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:27:47.136236 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fafe6-0b48-4dd2-b80b-8cbf5c314d2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:27:47.144395 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880fafe6-0b48-4dd2-b80b-8cbf5c314d2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:27:47.145834 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880fafe6-0b48-4dd2-b80b-8cbf5c314d2c service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:27:47.155160 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1515/3002] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:27:47 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:27:48.242182 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ea9bd-11aa-4ae5-aeee-ee47e8499a1b None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:27:48.242182 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ea9bd-11aa-4ae5-aeee-ee47e8499a1b None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:27:48.242182 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ea9bd-11aa-4ae5-aeee-ee47e8499a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:27:48.262176 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-535ea9bd-11aa-4ae5-aeee-ee47e8499a1b None None] MFA Rules not processed for user `f92d948810ea4241810473506b654f7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:27:48.264911 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-535ea9bd-11aa-4ae5-aeee-ee47e8499a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:27:48.274292 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1488/3003] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Jul 18 07:27:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:27:48.290592 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880fafe6-0b48-4dd2-b80b-8cbf5c314d2c service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:27:48.305383 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880fafe6-0b48-4dd2-b80b-8cbf5c314d2c service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:27:48.312753 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad8b138-5661-4d52-8f4b-de1c33bd0371 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:27:48.313690 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8b138-5661-4d52-8f4b-de1c33bd0371 service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:27:48.313985 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8b138-5661-4d52-8f4b-de1c33bd0371 service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:27:48.314267 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8b138-5661-4d52-8f4b-de1c33bd0371 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:27:48.325494 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad8b138-5661-4d52-8f4b-de1c33bd0371 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:27:48.327989 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad8b138-5661-4d52-8f4b-de1c33bd0371 service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:27:48.338949 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1516/3004] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:27:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:27:50.909769 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721f3f0-8958-4081-9bfc-c38000efe529 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:27:50.910262 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721f3f0-8958-4081-9bfc-c38000efe529 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:27:50.910466 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721f3f0-8958-4081-9bfc-c38000efe529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:27:50.928822 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-0721f3f0-8958-4081-9bfc-c38000efe529 None None] MFA Rules not processed for user `f92d948810ea4241810473506b654f7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:27:50.930059 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-0721f3f0-8958-4081-9bfc-c38000efe529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:27:50.940251 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1489/3005] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Jul 18 07:27:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:27:50.951900 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad8b138-5661-4d52-8f4b-de1c33bd0371 service nova] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:27:50.963443 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad8b138-5661-4d52-8f4b-de1c33bd0371 service nova] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:27:50.969161 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5cdb5c-743c-447a-801e-dda1bef363f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92d948810ea4241810473506b654f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:27:50.969796 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5cdb5c-743c-447a-801e-dda1bef363f5 service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:27:50.970084 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5cdb5c-743c-447a-801e-dda1bef363f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:27:50.970362 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5cdb5c-743c-447a-801e-dda1bef363f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:27:50.979145 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5cdb5c-743c-447a-801e-dda1bef363f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:27:50.981508 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5cdb5c-743c-447a-801e-dda1bef363f5 service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:27:50.992233 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1517/3006] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:27:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 07:27:53.607966 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0721f3f0-8958-4081-9bfc-c38000efe529 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:27:53.617232 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0721f3f0-8958-4081-9bfc-c38000efe529 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:27:53.621477 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5f613a-a619-4849-935a-2c65af34f7d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9824a96e2bcd4a16bfc6fe6bf4ea1ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:27:53.624460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f613a-a619-4849-935a-2c65af34f7d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:27:53.624460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f613a-a619-4849-935a-2c65af34f7d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:27:53.624460 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f613a-a619-4849-935a-2c65af34f7d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:27:53.685766 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5f613a-a619-4849-935a-2c65af34f7d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:27:53.687193 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5f613a-a619-4849-935a-2c65af34f7d3 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:27:53.697350 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1490/3007] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:27:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:27:57.089924 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4b400-1430-44fb-ad51-eba26ac49781 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:27:57.090470 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4b400-1430-44fb-ad51-eba26ac49781 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:27:57.090606 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4b400-1430-44fb-ad51-eba26ac49781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:27:57.127689 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-13a4b400-1430-44fb-ad51-eba26ac49781 None None] MFA Rules not processed for user `a3560534427544cdabfd0ba702a0a165`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 07:27:57.128972 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-13a4b400-1430-44fb-ad51-eba26ac49781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:27:57.138412 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1518/3008] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Jul 18 07:27:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 07:27:57.149529 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5f613a-a619-4849-935a-2c65af34f7d3 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:27:57.160783 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5f613a-a619-4849-935a-2c65af34f7d3 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:27:57.165828 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e25bd3-8960-4a75-bd41-66030cb2baa7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3560534427544cdabfd0ba702a0a165', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:27:57.166463 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e25bd3-8960-4a75-bd41-66030cb2baa7 service placement] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:27:57.166751 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e25bd3-8960-4a75-bd41-66030cb2baa7 service placement] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:27:57.167062 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e25bd3-8960-4a75-bd41-66030cb2baa7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:27:57.175531 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e25bd3-8960-4a75-bd41-66030cb2baa7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:27:57.177138 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e25bd3-8960-4a75-bd41-66030cb2baa7 service placement] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:27:57.187492 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1491/3009] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:27:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:28:00.586411 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4b400-1430-44fb-ad51-eba26ac49781 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:28:00.599131 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4b400-1430-44fb-ad51-eba26ac49781 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:28:00.603997 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5260a57-07ee-4811-9b74-2017e0aa1f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:28:00.604905 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5260a57-07ee-4811-9b74-2017e0aa1f0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:28:00.605192 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5260a57-07ee-4811-9b74-2017e0aa1f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:28:00.605473 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5260a57-07ee-4811-9b74-2017e0aa1f0a admin admin] PATH_INFO: `/v3/users/ece0f77dbbf1474b8f2205c2cbc7b881` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:28:00.606704 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5260a57-07ee-4811-9b74-2017e0aa1f0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ece0f77dbbf1474b8f2205c2cbc7b881)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:28:00.608869 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5260a57-07ee-4811-9b74-2017e0aa1f0a admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:28:00.786759 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d5260a57-07ee-4811-9b74-2017e0aa1f0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ece0f77dbbf1474b8f2205c2cbc7b881'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:28:00.793678 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d5260a57-07ee-4811-9b74-2017e0aa1f0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ece0f77dbbf1474b8f2205c2cbc7b881'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:28:00.794336 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d5260a57-07ee-4811-9b74-2017e0aa1f0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ece0f77dbbf1474b8f2205c2cbc7b881'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:28:00.794985 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d5260a57-07ee-4811-9b74-2017e0aa1f0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ece0f77dbbf1474b8f2205c2cbc7b881'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:28:00.806677 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d5260a57-07ee-4811-9b74-2017e0aa1f0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ece0f77dbbf1474b8f2205c2cbc7b881'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:28:00.904043 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1519/3010] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 07:28:00 2022] DELETE /identity/v3/users/ece0f77dbbf1474b8f2205c2cbc7b881 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:28:01.129416 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e25bd3-8960-4a75-bd41-66030cb2baa7 service placement] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:28:01.141011 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e25bd3-8960-4a75-bd41-66030cb2baa7 service placement] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:28:01.199133 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edff536-3193-4b8f-aa88-1d96a439c21a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:28:01.199986 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edff536-3193-4b8f-aa88-1d96a439c21a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:28:01.200271 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edff536-3193-4b8f-aa88-1d96a439c21a admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:28:01.200582 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edff536-3193-4b8f-aa88-1d96a439c21a admin admin] PATH_INFO: `/v3/projects/1acd582259d447829d53a5719dd31266` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:28:01.201747 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edff536-3193-4b8f-aa88-1d96a439c21a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1acd582259d447829d53a5719dd31266)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:28:01.203626 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edff536-3193-4b8f-aa88-1d96a439c21a admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:28:01.328240 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-7edff536-3193-4b8f-aa88-1d96a439c21a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1acd582259d447829d53a5719dd31266', 'request_id': 'req-7edff536-3193-4b8f-aa88-1d96a439c21a'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:28:01.333737 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-7edff536-3193-4b8f-aa88-1d96a439c21a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1acd582259d447829d53a5719dd31266', 'request_id': 'req-7edff536-3193-4b8f-aa88-1d96a439c21a'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:28:01.406906 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-7edff536-3193-4b8f-aa88-1d96a439c21a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1acd582259d447829d53a5719dd31266', 'request_id': 'req-7edff536-3193-4b8f-aa88-1d96a439c21a'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:28:01.414072 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:28:01.419299 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-7edff536-3193-4b8f-aa88-1d96a439c21a admin admin] The token cache is being invalidate because project 1acd582259d447829d53a5719dd31266 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 07:28:01.424547 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-7edff536-3193-4b8f-aa88-1d96a439c21a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:28:01.424547 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-7edff536-3193-4b8f-aa88-1d96a439c21a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:28:01.424547 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1492/3011] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 07:28:01 2022] DELETE /identity/v3/projects/1acd582259d447829d53a5719dd31266 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:28:01.425832 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5260a57-07ee-4811-9b74-2017e0aa1f0a admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:28:01.427227 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d5260a57-07ee-4811-9b74-2017e0aa1f0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:28:01.441438 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5260a57-07ee-4811-9b74-2017e0aa1f0a admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:28:01.499088 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51899b2-5fbd-48f2-925e-f9820ed69e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:28:01.499978 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51899b2-5fbd-48f2-925e-f9820ed69e24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:28:01.500272 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51899b2-5fbd-48f2-925e-f9820ed69e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:28:01.500586 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51899b2-5fbd-48f2-925e-f9820ed69e24 admin admin] PATH_INFO: `/v3/users/fdd45c60e092432a8f97b521246aa229` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:28:01.501840 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51899b2-5fbd-48f2-925e-f9820ed69e24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdd45c60e092432a8f97b521246aa229)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:28:01.503863 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51899b2-5fbd-48f2-925e-f9820ed69e24 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:28:01.517511 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:28:01.702419 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-a51899b2-5fbd-48f2-925e-f9820ed69e24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdd45c60e092432a8f97b521246aa229'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:28:01.709476 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-a51899b2-5fbd-48f2-925e-f9820ed69e24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdd45c60e092432a8f97b521246aa229'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:28:01.710173 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-a51899b2-5fbd-48f2-925e-f9820ed69e24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdd45c60e092432a8f97b521246aa229'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:28:01.710780 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-a51899b2-5fbd-48f2-925e-f9820ed69e24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdd45c60e092432a8f97b521246aa229'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:28:01.722533 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-a51899b2-5fbd-48f2-925e-f9820ed69e24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdd45c60e092432a8f97b521246aa229'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:28:01.815772 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1520/3012] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 18 07:28:01 2022] DELETE /identity/v3/users/fdd45c60e092432a8f97b521246aa229 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:28:02.054416 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edff536-3193-4b8f-aa88-1d96a439c21a admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:28:02.056173 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7edff536-3193-4b8f-aa88-1d96a439c21a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:28:02.070672 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edff536-3193-4b8f-aa88-1d96a439c21a admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:28:02.129098 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb218835-03c1-4c99-a3ca-c69993f0d071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8783051891e642af84b75a46b4d6b15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e19cd5e83fe45f5a260277527d6bf8c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:28:02.129952 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb218835-03c1-4c99-a3ca-c69993f0d071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:28:02.130515 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb218835-03c1-4c99-a3ca-c69993f0d071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:28:02.130515 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb218835-03c1-4c99-a3ca-c69993f0d071 admin admin] PATH_INFO: `/v3/projects/55849460624d495fa2c415ade0e707a2` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:28:02.131658 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb218835-03c1-4c99-a3ca-c69993f0d071 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55849460624d495fa2c415ade0e707a2)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:28:02.133577 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb218835-03c1-4c99-a3ca-c69993f0d071 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:28:02.230406 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-cb218835-03c1-4c99-a3ca-c69993f0d071 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55849460624d495fa2c415ade0e707a2', 'request_id': 'req-cb218835-03c1-4c99-a3ca-c69993f0d071'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:28:02.236711 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-cb218835-03c1-4c99-a3ca-c69993f0d071 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55849460624d495fa2c415ade0e707a2', 'request_id': 'req-cb218835-03c1-4c99-a3ca-c69993f0d071'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:28:02.315861 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-cb218835-03c1-4c99-a3ca-c69993f0d071 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55849460624d495fa2c415ade0e707a2', 'request_id': 'req-cb218835-03c1-4c99-a3ca-c69993f0d071'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:28:02.327792 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-cb218835-03c1-4c99-a3ca-c69993f0d071 admin admin] The token cache is being invalidate because project 55849460624d495fa2c415ade0e707a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 07:28:02.328077 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-cb218835-03c1-4c99-a3ca-c69993f0d071 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:28:02.328713 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-cb218835-03c1-4c99-a3ca-c69993f0d071 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 07:28:02.330240 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1493/3013] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 18 07:28:02 2022] DELETE /identity/v3/projects/55849460624d495fa2c415ade0e707a2 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 07:28:11.882025 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=21,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:28:12.324227 dv-cinder-850015-4-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=52,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 07:32:09.132126 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51899b2-5fbd-48f2-925e-f9820ed69e24 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 07:32:09.133752 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-a51899b2-5fbd-48f2-925e-f9820ed69e24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:32:09.254693 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51899b2-5fbd-48f2-925e-f9820ed69e24 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 07:32:09.312288 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ed870-9997-4705-be33-1a51ae12cab0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3560534427544cdabfd0ba702a0a165', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ed50a8a3e184bde9269d7b0d9573d4a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 07:32:09.313040 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ed870-9997-4705-be33-1a51ae12cab0 service placement] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 07:32:09.313330 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ed870-9997-4705-be33-1a51ae12cab0 service placement] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 07:32:09.313613 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ed870-9997-4705-be33-1a51ae12cab0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 07:32:09.315518 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-163ed870-9997-4705-be33-1a51ae12cab0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 07:32:09.327271 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163ed870-9997-4705-be33-1a51ae12cab0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 07:32:09.329032 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163ed870-9997-4705-be33-1a51ae12cab0 service placement] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 07:32:09.338681 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1521/3014] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 18 07:32:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 07:32:19.334546 dv-cinder-850015-4-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=81 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}